2023-05-04 10:53:38,754 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-05-04 10:53:39,029 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462 2023-05-04 10:53:39,044 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-05-04 10:53:39,045 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/cluster_3e5b0dbc-215f-67e3-28fd-7825f44cb1a4, deleteOnExit=true 2023-05-04 10:53:39,045 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-05-04 10:53:39,046 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/test.cache.data in system properties and HBase conf 2023-05-04 10:53:39,046 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/hadoop.tmp.dir in system properties and HBase conf 2023-05-04 10:53:39,046 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/hadoop.log.dir in system properties and HBase conf 2023-05-04 10:53:39,047 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/mapreduce.cluster.local.dir in system properties and HBase conf 2023-05-04 10:53:39,047 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-05-04 10:53:39,047 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-05-04 10:53:39,176 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-05-04 10:53:39,618 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-05-04 10:53:39,624 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-05-04 10:53:39,624 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-05-04 10:53:39,624 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-05-04 10:53:39,625 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-05-04 10:53:39,625 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-05-04 10:53:39,626 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-05-04 10:53:39,626 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-05-04 10:53:39,626 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/dfs.journalnode.edits.dir in system properties and HBase conf 2023-05-04 10:53:39,627 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-05-04 10:53:39,627 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/nfs.dump.dir in system properties and HBase conf 2023-05-04 10:53:39,628 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/java.io.tmpdir in system properties and HBase conf 2023-05-04 10:53:39,628 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/dfs.journalnode.edits.dir in system properties and HBase conf 2023-05-04 10:53:39,628 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-05-04 10:53:39,629 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-05-04 10:53:40,315 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-05-04 10:53:40,319 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-05-04 10:53:40,674 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-05-04 10:53:40,883 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-05-04 10:53:40,906 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-05-04 10:53:40,952 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-05-04 10:53:41,001 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/java.io.tmpdir/Jetty_localhost_35755_hdfs____.tj7ie9/webapp 2023-05-04 10:53:41,182 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35755 2023-05-04 10:53:41,205 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-05-04 10:53:41,205 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-05-04 10:53:41,832 WARN [Listener at localhost/44245] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-05-04 10:53:41,944 WARN [Listener at localhost/44245] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-05-04 10:53:41,971 WARN [Listener at localhost/44245] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-05-04 10:53:41,979 INFO [Listener at localhost/44245] log.Slf4jLog(67): jetty-6.1.26 2023-05-04 10:53:41,987 INFO [Listener at localhost/44245] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/java.io.tmpdir/Jetty_localhost_33707_datanode____yyf6dh/webapp 2023-05-04 10:53:42,124 INFO [Listener at localhost/44245] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33707 2023-05-04 10:53:42,501 WARN [Listener at localhost/39247] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-05-04 10:53:42,522 WARN [Listener at localhost/39247] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-05-04 10:53:42,528 WARN [Listener at localhost/39247] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-05-04 10:53:42,530 INFO [Listener at localhost/39247] log.Slf4jLog(67): jetty-6.1.26 2023-05-04 10:53:42,541 INFO [Listener at localhost/39247] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/java.io.tmpdir/Jetty_localhost_45637_datanode____.9kbfkw/webapp 2023-05-04 10:53:42,675 INFO [Listener at localhost/39247] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45637 2023-05-04 10:53:42,692 WARN [Listener at localhost/34261] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-05-04 10:53:42,729 WARN [Listener at localhost/34261] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-05-04 10:53:42,740 WARN [Listener at localhost/34261] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-05-04 10:53:42,742 INFO [Listener at localhost/34261] log.Slf4jLog(67): jetty-6.1.26 2023-05-04 10:53:42,749 INFO [Listener at localhost/34261] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/java.io.tmpdir/Jetty_localhost_37287_datanode____mq5x6c/webapp 2023-05-04 10:53:42,880 INFO [Listener at localhost/34261] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37287 2023-05-04 10:53:42,894 WARN [Listener at localhost/35483] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-05-04 10:53:43,146 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x42c18d3a8342afdc: Processing first storage report for DS-0b13d23a-a532-49b1-93c9-373b659adc8e from datanode 9432e34f-f582-4227-a602-1b8f52f4fa10 2023-05-04 10:53:43,148 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x42c18d3a8342afdc: from storage DS-0b13d23a-a532-49b1-93c9-373b659adc8e node DatanodeRegistration(127.0.0.1:38321, datanodeUuid=9432e34f-f582-4227-a602-1b8f52f4fa10, infoPort=42609, infoSecurePort=0, ipcPort=39247, storageInfo=lv=-57;cid=testClusterID;nsid=1470550737;c=1683197620399), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-05-04 10:53:43,148 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x713fb045d60eda5f: Processing first storage report for DS-ff8dcb33-2008-4522-bdfb-c059988b1bb3 from datanode 4e8a05ef-1582-45ea-8d3b-86bbf8f33f91 2023-05-04 10:53:43,148 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x713fb045d60eda5f: from storage DS-ff8dcb33-2008-4522-bdfb-c059988b1bb3 node DatanodeRegistration(127.0.0.1:44687, datanodeUuid=4e8a05ef-1582-45ea-8d3b-86bbf8f33f91, infoPort=43053, infoSecurePort=0, ipcPort=34261, storageInfo=lv=-57;cid=testClusterID;nsid=1470550737;c=1683197620399), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-04 10:53:43,148 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x73ab35c7767bd251: Processing first storage report for DS-e2fcb6ee-ea17-48bd-ab8e-961f13d8341b from datanode eabd0028-d500-4e17-b99c-cf8614a5876b 2023-05-04 10:53:43,149 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x73ab35c7767bd251: from storage DS-e2fcb6ee-ea17-48bd-ab8e-961f13d8341b node DatanodeRegistration(127.0.0.1:34481, datanodeUuid=eabd0028-d500-4e17-b99c-cf8614a5876b, infoPort=36205, infoSecurePort=0, ipcPort=35483, storageInfo=lv=-57;cid=testClusterID;nsid=1470550737;c=1683197620399), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-04 10:53:43,149 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x42c18d3a8342afdc: Processing first storage report for DS-1613c021-80fc-48fa-806c-d88da58dd20a from datanode 9432e34f-f582-4227-a602-1b8f52f4fa10 2023-05-04 10:53:43,149 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x42c18d3a8342afdc: from storage DS-1613c021-80fc-48fa-806c-d88da58dd20a node DatanodeRegistration(127.0.0.1:38321, datanodeUuid=9432e34f-f582-4227-a602-1b8f52f4fa10, infoPort=42609, infoSecurePort=0, ipcPort=39247, storageInfo=lv=-57;cid=testClusterID;nsid=1470550737;c=1683197620399), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-04 10:53:43,149 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x713fb045d60eda5f: Processing first storage report for DS-8c6e7018-7ffe-47e4-9396-d38d0dd7a62b from datanode 4e8a05ef-1582-45ea-8d3b-86bbf8f33f91 2023-05-04 10:53:43,149 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x713fb045d60eda5f: from storage DS-8c6e7018-7ffe-47e4-9396-d38d0dd7a62b node DatanodeRegistration(127.0.0.1:44687, datanodeUuid=4e8a05ef-1582-45ea-8d3b-86bbf8f33f91, infoPort=43053, infoSecurePort=0, ipcPort=34261, storageInfo=lv=-57;cid=testClusterID;nsid=1470550737;c=1683197620399), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-04 10:53:43,149 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x73ab35c7767bd251: Processing first storage report for DS-cef8487b-ef87-40f7-82f3-966bee3fee43 from datanode eabd0028-d500-4e17-b99c-cf8614a5876b 2023-05-04 10:53:43,149 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x73ab35c7767bd251: from storage DS-cef8487b-ef87-40f7-82f3-966bee3fee43 node DatanodeRegistration(127.0.0.1:34481, datanodeUuid=eabd0028-d500-4e17-b99c-cf8614a5876b, infoPort=36205, infoSecurePort=0, ipcPort=35483, storageInfo=lv=-57;cid=testClusterID;nsid=1470550737;c=1683197620399), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-04 10:53:43,329 DEBUG [Listener at localhost/35483] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462 2023-05-04 10:53:43,403 INFO [Listener at localhost/35483] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/cluster_3e5b0dbc-215f-67e3-28fd-7825f44cb1a4/zookeeper_0, clientPort=53687, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/cluster_3e5b0dbc-215f-67e3-28fd-7825f44cb1a4/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/cluster_3e5b0dbc-215f-67e3-28fd-7825f44cb1a4/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-05-04 10:53:43,418 INFO [Listener at localhost/35483] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=53687 2023-05-04 10:53:43,426 INFO [Listener at localhost/35483] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-04 10:53:43,428 INFO [Listener at localhost/35483] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-04 10:53:44,115 INFO [Listener at localhost/35483] util.FSUtils(471): Created version file at hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d with version=8 2023-05-04 10:53:44,115 INFO [Listener at localhost/35483] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/hbase-staging 2023-05-04 10:53:44,453 INFO [Listener at localhost/35483] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-05-04 10:53:44,907 INFO [Listener at localhost/35483] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-05-04 10:53:44,938 INFO [Listener at localhost/35483] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-04 10:53:44,939 INFO [Listener at localhost/35483] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-05-04 10:53:44,939 INFO [Listener at localhost/35483] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-05-04 10:53:44,939 INFO [Listener at localhost/35483] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-04 10:53:44,940 INFO [Listener at localhost/35483] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-05-04 10:53:45,081 INFO [Listener at localhost/35483] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-05-04 10:53:45,158 DEBUG [Listener at localhost/35483] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-05-04 10:53:45,251 INFO [Listener at localhost/35483] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40233 2023-05-04 10:53:45,261 INFO [Listener at localhost/35483] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-04 10:53:45,263 INFO [Listener at localhost/35483] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-04 10:53:45,283 INFO [Listener at localhost/35483] zookeeper.RecoverableZooKeeper(93): Process identifier=master:40233 connecting to ZooKeeper ensemble=127.0.0.1:53687 2023-05-04 10:53:45,326 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:402330x0, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-05-04 10:53:45,329 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:40233-0x101e94d661e0000 connected 2023-05-04 10:53:45,353 DEBUG [Listener at localhost/35483] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-04 10:53:45,353 DEBUG [Listener at localhost/35483] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-04 10:53:45,357 DEBUG [Listener at localhost/35483] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-05-04 10:53:45,365 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40233 2023-05-04 10:53:45,365 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40233 2023-05-04 10:53:45,366 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40233 2023-05-04 10:53:45,366 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40233 2023-05-04 10:53:45,366 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40233 2023-05-04 10:53:45,372 INFO [Listener at localhost/35483] master.HMaster(444): hbase.rootdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d, hbase.cluster.distributed=false 2023-05-04 10:53:45,439 INFO [Listener at localhost/35483] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-05-04 10:53:45,439 INFO [Listener at localhost/35483] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-04 10:53:45,439 INFO [Listener at localhost/35483] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-05-04 10:53:45,439 INFO [Listener at localhost/35483] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-05-04 10:53:45,439 INFO [Listener at localhost/35483] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-04 10:53:45,440 INFO [Listener at localhost/35483] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-05-04 10:53:45,444 INFO [Listener at localhost/35483] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-05-04 10:53:45,447 INFO [Listener at localhost/35483] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37343 2023-05-04 10:53:45,449 INFO [Listener at localhost/35483] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-05-04 10:53:45,455 DEBUG [Listener at localhost/35483] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-05-04 10:53:45,457 INFO [Listener at localhost/35483] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-04 10:53:45,458 INFO [Listener at localhost/35483] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-04 10:53:45,460 INFO [Listener at localhost/35483] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37343 connecting to ZooKeeper ensemble=127.0.0.1:53687 2023-05-04 10:53:45,468 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:373430x0, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-05-04 10:53:45,469 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:37343-0x101e94d661e0001 connected 2023-05-04 10:53:45,469 DEBUG [Listener at localhost/35483] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-04 10:53:45,470 DEBUG [Listener at localhost/35483] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-04 10:53:45,471 DEBUG [Listener at localhost/35483] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-05-04 10:53:45,476 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37343 2023-05-04 10:53:45,476 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37343 2023-05-04 10:53:45,476 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37343 2023-05-04 10:53:45,477 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37343 2023-05-04 10:53:45,477 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37343 2023-05-04 10:53:45,490 INFO [Listener at localhost/35483] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-05-04 10:53:45,490 INFO [Listener at localhost/35483] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-04 10:53:45,490 INFO [Listener at localhost/35483] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-05-04 10:53:45,491 INFO [Listener at localhost/35483] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-05-04 10:53:45,491 INFO [Listener at localhost/35483] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-04 10:53:45,491 INFO [Listener at localhost/35483] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-05-04 10:53:45,491 INFO [Listener at localhost/35483] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-05-04 10:53:45,493 INFO [Listener at localhost/35483] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33581 2023-05-04 10:53:45,493 INFO [Listener at localhost/35483] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-05-04 10:53:45,496 DEBUG [Listener at localhost/35483] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-05-04 10:53:45,496 INFO [Listener at localhost/35483] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-04 10:53:45,499 INFO [Listener at localhost/35483] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-04 10:53:45,502 INFO [Listener at localhost/35483] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33581 connecting to ZooKeeper ensemble=127.0.0.1:53687 2023-05-04 10:53:45,507 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:335810x0, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-05-04 10:53:45,508 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:33581-0x101e94d661e0002 connected 2023-05-04 10:53:45,508 DEBUG [Listener at localhost/35483] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-04 10:53:45,509 DEBUG [Listener at localhost/35483] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-04 10:53:45,510 DEBUG [Listener at localhost/35483] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-05-04 10:53:45,511 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33581 2023-05-04 10:53:45,512 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33581 2023-05-04 10:53:45,512 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33581 2023-05-04 10:53:45,516 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33581 2023-05-04 10:53:45,516 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33581 2023-05-04 10:53:45,529 INFO [Listener at localhost/35483] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-05-04 10:53:45,529 INFO [Listener at localhost/35483] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-04 10:53:45,529 INFO [Listener at localhost/35483] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-05-04 10:53:45,529 INFO [Listener at localhost/35483] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-05-04 10:53:45,529 INFO [Listener at localhost/35483] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-04 10:53:45,530 INFO [Listener at localhost/35483] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-05-04 10:53:45,530 INFO [Listener at localhost/35483] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-05-04 10:53:45,531 INFO [Listener at localhost/35483] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42285 2023-05-04 10:53:45,532 INFO [Listener at localhost/35483] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-05-04 10:53:45,533 DEBUG [Listener at localhost/35483] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-05-04 10:53:45,535 INFO [Listener at localhost/35483] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-04 10:53:45,537 INFO [Listener at localhost/35483] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-04 10:53:45,538 INFO [Listener at localhost/35483] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42285 connecting to ZooKeeper ensemble=127.0.0.1:53687 2023-05-04 10:53:45,542 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:422850x0, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-05-04 10:53:45,543 DEBUG [Listener at localhost/35483] zookeeper.ZKUtil(164): regionserver:422850x0, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-04 10:53:45,543 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42285-0x101e94d661e0003 connected 2023-05-04 10:53:45,544 DEBUG [Listener at localhost/35483] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-04 10:53:45,545 DEBUG [Listener at localhost/35483] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-05-04 10:53:45,548 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42285 2023-05-04 10:53:45,548 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42285 2023-05-04 10:53:45,549 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42285 2023-05-04 10:53:45,549 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42285 2023-05-04 10:53:45,549 DEBUG [Listener at localhost/35483] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42285 2023-05-04 10:53:45,553 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,40233,1683197624291 2023-05-04 10:53:45,562 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-05-04 10:53:45,563 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,40233,1683197624291 2023-05-04 10:53:45,584 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-05-04 10:53:45,584 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-05-04 10:53:45,584 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-05-04 10:53:45,585 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-04 10:53:45,584 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-05-04 10:53:45,586 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-05-04 10:53:45,588 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,40233,1683197624291 from backup master directory 2023-05-04 10:53:45,588 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-05-04 10:53:45,592 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,40233,1683197624291 2023-05-04 10:53:45,592 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-05-04 10:53:45,593 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-05-04 10:53:45,593 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,40233,1683197624291 2023-05-04 10:53:45,597 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-05-04 10:53:45,599 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-05-04 10:53:45,694 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/hbase.id with ID: be1bcbb5-b05f-4e1f-9547-c8f3feb0d046 2023-05-04 10:53:45,739 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-04 10:53:45,755 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-04 10:53:45,813 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x2fa7d594 to 127.0.0.1:53687 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-04 10:53:45,846 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@212ab076, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-04 10:53:45,869 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-05-04 10:53:45,871 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-05-04 10:53:45,888 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-05-04 10:53:45,888 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-05-04 10:53:45,890 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-05-04 10:53:45,894 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-05-04 10:53:45,895 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-04 10:53:45,929 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/MasterData/data/master/store-tmp 2023-05-04 10:53:45,982 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:53:45,983 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-05-04 10:53:45,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-04 10:53:45,983 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-04 10:53:45,983 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-05-04 10:53:45,983 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-04 10:53:45,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-04 10:53:45,983 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-05-04 10:53:45,985 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/MasterData/WALs/jenkins-hbase4.apache.org,40233,1683197624291 2023-05-04 10:53:46,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40233%2C1683197624291, suffix=, logDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/MasterData/WALs/jenkins-hbase4.apache.org,40233,1683197624291, archiveDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/MasterData/oldWALs, maxLogs=10 2023-05-04 10:53:46,061 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38321,DS-0b13d23a-a532-49b1-93c9-373b659adc8e,DISK] 2023-05-04 10:53:46,061 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44687,DS-ff8dcb33-2008-4522-bdfb-c059988b1bb3,DISK] 2023-05-04 10:53:46,061 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34481,DS-e2fcb6ee-ea17-48bd-ab8e-961f13d8341b,DISK] 2023-05-04 10:53:46,069 DEBUG [RS-EventLoopGroup-5-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-05-04 10:53:46,141 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/MasterData/WALs/jenkins-hbase4.apache.org,40233,1683197624291/jenkins-hbase4.apache.org%2C40233%2C1683197624291.1683197626014 2023-05-04 10:53:46,144 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44687,DS-ff8dcb33-2008-4522-bdfb-c059988b1bb3,DISK], DatanodeInfoWithStorage[127.0.0.1:34481,DS-e2fcb6ee-ea17-48bd-ab8e-961f13d8341b,DISK], DatanodeInfoWithStorage[127.0.0.1:38321,DS-0b13d23a-a532-49b1-93c9-373b659adc8e,DISK]] 2023-05-04 10:53:46,145 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-05-04 10:53:46,145 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:53:46,149 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-05-04 10:53:46,150 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-05-04 10:53:46,206 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-05-04 10:53:46,212 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-05-04 10:53:46,236 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-05-04 10:53:46,248 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:46,254 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-05-04 10:53:46,255 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-05-04 10:53:46,270 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-05-04 10:53:46,274 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-04 10:53:46,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74275598, jitterRate=0.10679265856742859}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-04 10:53:46,275 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-05-04 10:53:46,276 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-05-04 10:53:46,297 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:53:46,297 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:53:46,299 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:53:46,301 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-05-04 10:53:46,334 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 32 msec 2023-05-04 10:53:46,334 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-05-04 10:53:46,358 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-05-04 10:53:46,364 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-05-04 10:53:46,387 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-05-04 10:53:46,391 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-05-04 10:53:46,392 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-05-04 10:53:46,397 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-05-04 10:53:46,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-05-04 10:53:46,405 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-04 10:53:46,406 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-05-04 10:53:46,407 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-05-04 10:53:46,419 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-05-04 10:53:46,424 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-05-04 10:53:46,424 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-05-04 10:53:46,424 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-04 10:53:46,424 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-05-04 10:53:46,424 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-05-04 10:53:46,425 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,40233,1683197624291, sessionid=0x101e94d661e0000, setting cluster-up flag (Was=false) 2023-05-04 10:53:46,440 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-04 10:53:46,446 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-05-04 10:53:46,447 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,40233,1683197624291 2023-05-04 10:53:46,456 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-04 10:53:46,462 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-05-04 10:53:46,463 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,40233,1683197624291 2023-05-04 10:53:46,466 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.hbase-snapshot/.tmp 2023-05-04 10:53:46,555 INFO [RS:2;jenkins-hbase4:42285] regionserver.HRegionServer(952): ClusterId : be1bcbb5-b05f-4e1f-9547-c8f3feb0d046 2023-05-04 10:53:46,556 INFO [RS:1;jenkins-hbase4:33581] regionserver.HRegionServer(952): ClusterId : be1bcbb5-b05f-4e1f-9547-c8f3feb0d046 2023-05-04 10:53:46,555 INFO [RS:0;jenkins-hbase4:37343] regionserver.HRegionServer(952): ClusterId : be1bcbb5-b05f-4e1f-9547-c8f3feb0d046 2023-05-04 10:53:46,562 DEBUG [RS:0;jenkins-hbase4:37343] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-05-04 10:53:46,562 DEBUG [RS:1;jenkins-hbase4:33581] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-05-04 10:53:46,562 DEBUG [RS:2;jenkins-hbase4:42285] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-05-04 10:53:46,569 DEBUG [RS:1;jenkins-hbase4:33581] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-05-04 10:53:46,569 DEBUG [RS:2;jenkins-hbase4:42285] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-05-04 10:53:46,569 DEBUG [RS:0;jenkins-hbase4:37343] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-05-04 10:53:46,569 DEBUG [RS:2;jenkins-hbase4:42285] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-05-04 10:53:46,569 DEBUG [RS:1;jenkins-hbase4:33581] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-05-04 10:53:46,569 DEBUG [RS:0;jenkins-hbase4:37343] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-05-04 10:53:46,574 DEBUG [RS:1;jenkins-hbase4:33581] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-05-04 10:53:46,574 DEBUG [RS:2;jenkins-hbase4:42285] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-05-04 10:53:46,575 DEBUG [RS:0;jenkins-hbase4:37343] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-05-04 10:53:46,576 DEBUG [RS:1;jenkins-hbase4:33581] zookeeper.ReadOnlyZKClient(139): Connect 0x1cabd420 to 127.0.0.1:53687 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-04 10:53:46,576 DEBUG [RS:2;jenkins-hbase4:42285] zookeeper.ReadOnlyZKClient(139): Connect 0x4da1b00c to 127.0.0.1:53687 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-04 10:53:46,576 DEBUG [RS:0;jenkins-hbase4:37343] zookeeper.ReadOnlyZKClient(139): Connect 0x66f3f124 to 127.0.0.1:53687 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-04 10:53:46,593 DEBUG [RS:1;jenkins-hbase4:33581] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@126c39c5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-04 10:53:46,594 DEBUG [RS:2;jenkins-hbase4:42285] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@436657c8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-04 10:53:46,594 DEBUG [RS:1;jenkins-hbase4:33581] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1653bf50, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-05-04 10:53:46,593 DEBUG [RS:0;jenkins-hbase4:37343] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6834038a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-04 10:53:46,594 DEBUG [RS:2;jenkins-hbase4:42285] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4759db7c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-05-04 10:53:46,595 DEBUG [RS:0;jenkins-hbase4:37343] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6c0b9a54, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-05-04 10:53:46,599 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-05-04 10:53:46,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-05-04 10:53:46,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-05-04 10:53:46,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-05-04 10:53:46,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-05-04 10:53:46,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-05-04 10:53:46,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-05-04 10:53:46,610 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,610 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197656610 2023-05-04 10:53:46,613 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-05-04 10:53:46,618 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-05-04 10:53:46,618 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-05-04 10:53:46,621 DEBUG [RS:2;jenkins-hbase4:42285] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:42285 2023-05-04 10:53:46,622 DEBUG [RS:0;jenkins-hbase4:37343] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:37343 2023-05-04 10:53:46,622 DEBUG [RS:1;jenkins-hbase4:33581] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:33581 2023-05-04 10:53:46,624 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-05-04 10:53:46,626 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-05-04 10:53:46,627 INFO [RS:2;jenkins-hbase4:42285] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-05-04 10:53:46,629 INFO [RS:2;jenkins-hbase4:42285] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-05-04 10:53:46,629 DEBUG [RS:2;jenkins-hbase4:42285] regionserver.HRegionServer(1023): About to register with Master. 2023-05-04 10:53:46,628 INFO [RS:0;jenkins-hbase4:37343] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-05-04 10:53:46,630 INFO [RS:0;jenkins-hbase4:37343] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-05-04 10:53:46,628 INFO [RS:1;jenkins-hbase4:33581] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-05-04 10:53:46,630 INFO [RS:1;jenkins-hbase4:33581] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-05-04 10:53:46,630 DEBUG [RS:0;jenkins-hbase4:37343] regionserver.HRegionServer(1023): About to register with Master. 2023-05-04 10:53:46,630 DEBUG [RS:1;jenkins-hbase4:33581] regionserver.HRegionServer(1023): About to register with Master. 2023-05-04 10:53:46,632 INFO [RS:1;jenkins-hbase4:33581] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,40233,1683197624291 with isa=jenkins-hbase4.apache.org/172.31.14.131:33581, startcode=1683197625489 2023-05-04 10:53:46,632 INFO [RS:0;jenkins-hbase4:37343] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,40233,1683197624291 with isa=jenkins-hbase4.apache.org/172.31.14.131:37343, startcode=1683197625438 2023-05-04 10:53:46,632 INFO [RS:2;jenkins-hbase4:42285] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,40233,1683197624291 with isa=jenkins-hbase4.apache.org/172.31.14.131:42285, startcode=1683197625528 2023-05-04 10:53:46,636 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-05-04 10:53:46,636 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-05-04 10:53:46,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-05-04 10:53:46,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-05-04 10:53:46,640 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,645 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-05-04 10:53:46,647 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-05-04 10:53:46,647 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-05-04 10:53:46,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-05-04 10:53:46,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-05-04 10:53:46,654 DEBUG [RS:1;jenkins-hbase4:33581] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-05-04 10:53:46,654 DEBUG [RS:2;jenkins-hbase4:42285] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-05-04 10:53:46,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1683197626657,5,FailOnTimeoutGroup] 2023-05-04 10:53:46,654 DEBUG [RS:0;jenkins-hbase4:37343] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-05-04 10:53:46,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1683197626658,5,FailOnTimeoutGroup] 2023-05-04 10:53:46,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-05-04 10:53:46,660 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,660 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,685 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-05-04 10:53:46,686 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-05-04 10:53:46,686 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d 2023-05-04 10:53:46,719 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:53:46,723 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-05-04 10:53:46,725 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/info 2023-05-04 10:53:46,726 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-05-04 10:53:46,727 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:46,727 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-05-04 10:53:46,730 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/rep_barrier 2023-05-04 10:53:46,730 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-05-04 10:53:46,731 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:46,732 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-05-04 10:53:46,732 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33991, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-05-04 10:53:46,732 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42387, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-05-04 10:53:46,732 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60361, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-05-04 10:53:46,734 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/table 2023-05-04 10:53:46,735 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-05-04 10:53:46,736 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:46,738 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740 2023-05-04 10:53:46,739 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740 2023-05-04 10:53:46,743 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-05-04 10:53:46,745 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-05-04 10:53:46,746 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:46,746 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40233] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:46,747 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40233] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:46,750 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-04 10:53:46,751 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71311275, jitterRate=0.06262080371379852}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-05-04 10:53:46,751 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-05-04 10:53:46,751 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-05-04 10:53:46,751 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-05-04 10:53:46,751 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-05-04 10:53:46,751 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-05-04 10:53:46,751 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-05-04 10:53:46,757 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-05-04 10:53:46,757 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-05-04 10:53:46,766 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-05-04 10:53:46,766 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-05-04 10:53:46,770 DEBUG [RS:1;jenkins-hbase4:33581] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d 2023-05-04 10:53:46,770 DEBUG [RS:0;jenkins-hbase4:37343] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d 2023-05-04 10:53:46,770 DEBUG [RS:2;jenkins-hbase4:42285] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d 2023-05-04 10:53:46,770 DEBUG [RS:0;jenkins-hbase4:37343] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:44245 2023-05-04 10:53:46,770 DEBUG [RS:1;jenkins-hbase4:33581] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:44245 2023-05-04 10:53:46,770 DEBUG [RS:0;jenkins-hbase4:37343] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-05-04 10:53:46,770 DEBUG [RS:2;jenkins-hbase4:42285] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:44245 2023-05-04 10:53:46,771 DEBUG [RS:2;jenkins-hbase4:42285] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-05-04 10:53:46,771 DEBUG [RS:1;jenkins-hbase4:33581] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-05-04 10:53:46,778 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-05-04 10:53:46,785 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-04 10:53:46,786 DEBUG [RS:1;jenkins-hbase4:33581] zookeeper.ZKUtil(162): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:46,786 DEBUG [RS:2;jenkins-hbase4:42285] zookeeper.ZKUtil(162): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:46,787 WARN [RS:1;jenkins-hbase4:33581] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-05-04 10:53:46,787 WARN [RS:2;jenkins-hbase4:42285] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-05-04 10:53:46,788 INFO [RS:1;jenkins-hbase4:33581] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-04 10:53:46,788 DEBUG [RS:0;jenkins-hbase4:37343] zookeeper.ZKUtil(162): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:46,788 INFO [RS:2;jenkins-hbase4:42285] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-04 10:53:46,788 DEBUG [RS:1;jenkins-hbase4:33581] regionserver.HRegionServer(1947): logDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/WALs/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:46,788 WARN [RS:0;jenkins-hbase4:37343] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-05-04 10:53:46,789 DEBUG [RS:2;jenkins-hbase4:42285] regionserver.HRegionServer(1947): logDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/WALs/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:46,789 INFO [RS:0;jenkins-hbase4:37343] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-04 10:53:46,789 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,37343,1683197625438] 2023-05-04 10:53:46,789 DEBUG [RS:0;jenkins-hbase4:37343] regionserver.HRegionServer(1947): logDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/WALs/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:46,789 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33581,1683197625489] 2023-05-04 10:53:46,789 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42285,1683197625528] 2023-05-04 10:53:46,801 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-05-04 10:53:46,804 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-05-04 10:53:46,804 DEBUG [RS:1;jenkins-hbase4:33581] zookeeper.ZKUtil(162): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:46,804 DEBUG [RS:2;jenkins-hbase4:42285] zookeeper.ZKUtil(162): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:46,804 DEBUG [RS:0;jenkins-hbase4:37343] zookeeper.ZKUtil(162): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:46,805 DEBUG [RS:1;jenkins-hbase4:33581] zookeeper.ZKUtil(162): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:46,806 DEBUG [RS:2;jenkins-hbase4:42285] zookeeper.ZKUtil(162): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:46,806 DEBUG [RS:0;jenkins-hbase4:37343] zookeeper.ZKUtil(162): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:46,806 DEBUG [RS:1;jenkins-hbase4:33581] zookeeper.ZKUtil(162): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:46,806 DEBUG [RS:2;jenkins-hbase4:42285] zookeeper.ZKUtil(162): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:46,807 DEBUG [RS:0;jenkins-hbase4:37343] zookeeper.ZKUtil(162): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:46,816 DEBUG [RS:1;jenkins-hbase4:33581] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-05-04 10:53:46,816 DEBUG [RS:0;jenkins-hbase4:37343] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-05-04 10:53:46,816 DEBUG [RS:2;jenkins-hbase4:42285] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-05-04 10:53:46,825 INFO [RS:1;jenkins-hbase4:33581] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-05-04 10:53:46,826 INFO [RS:2;jenkins-hbase4:42285] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-05-04 10:53:46,825 INFO [RS:0;jenkins-hbase4:37343] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-05-04 10:53:46,845 INFO [RS:0;jenkins-hbase4:37343] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-05-04 10:53:46,845 INFO [RS:2;jenkins-hbase4:42285] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-05-04 10:53:46,845 INFO [RS:1;jenkins-hbase4:33581] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-05-04 10:53:46,849 INFO [RS:1;jenkins-hbase4:33581] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-05-04 10:53:46,849 INFO [RS:0;jenkins-hbase4:37343] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-05-04 10:53:46,849 INFO [RS:2;jenkins-hbase4:42285] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-05-04 10:53:46,850 INFO [RS:0;jenkins-hbase4:37343] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,850 INFO [RS:1;jenkins-hbase4:33581] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,850 INFO [RS:2;jenkins-hbase4:42285] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,851 INFO [RS:0;jenkins-hbase4:37343] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-05-04 10:53:46,851 INFO [RS:2;jenkins-hbase4:42285] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-05-04 10:53:46,851 INFO [RS:1;jenkins-hbase4:33581] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-05-04 10:53:46,860 INFO [RS:0;jenkins-hbase4:37343] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,860 INFO [RS:1;jenkins-hbase4:33581] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,860 INFO [RS:2;jenkins-hbase4:42285] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,860 DEBUG [RS:0;jenkins-hbase4:37343] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,860 DEBUG [RS:1;jenkins-hbase4:33581] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,860 DEBUG [RS:2;jenkins-hbase4:42285] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,860 DEBUG [RS:1;jenkins-hbase4:33581] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,860 DEBUG [RS:2;jenkins-hbase4:42285] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,860 DEBUG [RS:1;jenkins-hbase4:33581] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,860 DEBUG [RS:2;jenkins-hbase4:42285] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,861 DEBUG [RS:1;jenkins-hbase4:33581] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,861 DEBUG [RS:2;jenkins-hbase4:42285] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,861 DEBUG [RS:1;jenkins-hbase4:33581] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,861 DEBUG [RS:2;jenkins-hbase4:42285] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,860 DEBUG [RS:0;jenkins-hbase4:37343] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,861 DEBUG [RS:2;jenkins-hbase4:42285] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-05-04 10:53:46,861 DEBUG [RS:0;jenkins-hbase4:37343] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,861 DEBUG [RS:2;jenkins-hbase4:42285] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,861 DEBUG [RS:0;jenkins-hbase4:37343] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,861 DEBUG [RS:2;jenkins-hbase4:42285] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,861 DEBUG [RS:1;jenkins-hbase4:33581] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-05-04 10:53:46,861 DEBUG [RS:2;jenkins-hbase4:42285] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,861 DEBUG [RS:1;jenkins-hbase4:33581] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,861 DEBUG [RS:2;jenkins-hbase4:42285] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,861 DEBUG [RS:1;jenkins-hbase4:33581] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,861 DEBUG [RS:0;jenkins-hbase4:37343] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,862 DEBUG [RS:1;jenkins-hbase4:33581] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,862 DEBUG [RS:0;jenkins-hbase4:37343] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-05-04 10:53:46,862 DEBUG [RS:1;jenkins-hbase4:33581] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,862 DEBUG [RS:0;jenkins-hbase4:37343] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,862 DEBUG [RS:0;jenkins-hbase4:37343] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,862 DEBUG [RS:0;jenkins-hbase4:37343] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,862 DEBUG [RS:0;jenkins-hbase4:37343] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-04 10:53:46,868 INFO [RS:2;jenkins-hbase4:42285] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,868 INFO [RS:2;jenkins-hbase4:42285] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,868 INFO [RS:1;jenkins-hbase4:33581] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,868 INFO [RS:2;jenkins-hbase4:42285] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,868 INFO [RS:1;jenkins-hbase4:33581] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,868 INFO [RS:0;jenkins-hbase4:37343] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,869 INFO [RS:1;jenkins-hbase4:33581] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,869 INFO [RS:0;jenkins-hbase4:37343] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,869 INFO [RS:0;jenkins-hbase4:37343] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,886 INFO [RS:2;jenkins-hbase4:42285] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-05-04 10:53:46,886 INFO [RS:1;jenkins-hbase4:33581] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-05-04 10:53:46,889 INFO [RS:2;jenkins-hbase4:42285] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42285,1683197625528-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,889 INFO [RS:0;jenkins-hbase4:37343] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-05-04 10:53:46,889 INFO [RS:1;jenkins-hbase4:33581] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33581,1683197625489-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,890 INFO [RS:0;jenkins-hbase4:37343] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37343,1683197625438-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:46,917 INFO [RS:0;jenkins-hbase4:37343] regionserver.Replication(203): jenkins-hbase4.apache.org,37343,1683197625438 started 2023-05-04 10:53:46,917 INFO [RS:1;jenkins-hbase4:33581] regionserver.Replication(203): jenkins-hbase4.apache.org,33581,1683197625489 started 2023-05-04 10:53:46,917 INFO [RS:0;jenkins-hbase4:37343] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,37343,1683197625438, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:37343, sessionid=0x101e94d661e0001 2023-05-04 10:53:46,917 INFO [RS:1;jenkins-hbase4:33581] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,33581,1683197625489, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33581, sessionid=0x101e94d661e0002 2023-05-04 10:53:46,918 INFO [RS:2;jenkins-hbase4:42285] regionserver.Replication(203): jenkins-hbase4.apache.org,42285,1683197625528 started 2023-05-04 10:53:46,919 DEBUG [RS:0;jenkins-hbase4:37343] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-05-04 10:53:46,919 INFO [RS:2;jenkins-hbase4:42285] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,42285,1683197625528, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42285, sessionid=0x101e94d661e0003 2023-05-04 10:53:46,919 DEBUG [RS:1;jenkins-hbase4:33581] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-05-04 10:53:46,919 DEBUG [RS:2;jenkins-hbase4:42285] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-05-04 10:53:46,919 DEBUG [RS:0;jenkins-hbase4:37343] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:46,919 DEBUG [RS:2;jenkins-hbase4:42285] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:46,920 DEBUG [RS:0;jenkins-hbase4:37343] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37343,1683197625438' 2023-05-04 10:53:46,919 DEBUG [RS:1;jenkins-hbase4:33581] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:46,920 DEBUG [RS:0;jenkins-hbase4:37343] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:46,920 DEBUG [RS:2;jenkins-hbase4:42285] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42285,1683197625528' 2023-05-04 10:53:46,920 DEBUG [RS:2;jenkins-hbase4:42285] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:46,920 DEBUG [RS:1;jenkins-hbase4:33581] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33581,1683197625489' 2023-05-04 10:53:46,920 DEBUG [RS:1;jenkins-hbase4:33581] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:46,921 DEBUG [RS:0;jenkins-hbase4:37343] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:46,921 DEBUG [RS:2;jenkins-hbase4:42285] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:46,921 DEBUG [RS:1;jenkins-hbase4:33581] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:46,922 DEBUG [RS:0;jenkins-hbase4:37343] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-05-04 10:53:46,922 DEBUG [RS:0;jenkins-hbase4:37343] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-05-04 10:53:46,922 DEBUG [RS:2;jenkins-hbase4:42285] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-05-04 10:53:46,922 DEBUG [RS:0;jenkins-hbase4:37343] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:46,922 DEBUG [RS:1;jenkins-hbase4:33581] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-05-04 10:53:46,922 DEBUG [RS:0;jenkins-hbase4:37343] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37343,1683197625438' 2023-05-04 10:53:46,922 DEBUG [RS:0;jenkins-hbase4:37343] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-05-04 10:53:46,922 DEBUG [RS:2;jenkins-hbase4:42285] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-05-04 10:53:46,922 DEBUG [RS:1;jenkins-hbase4:33581] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-05-04 10:53:46,923 DEBUG [RS:1;jenkins-hbase4:33581] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:46,922 DEBUG [RS:2;jenkins-hbase4:42285] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:46,923 DEBUG [RS:1;jenkins-hbase4:33581] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33581,1683197625489' 2023-05-04 10:53:46,923 DEBUG [RS:1;jenkins-hbase4:33581] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-05-04 10:53:46,923 DEBUG [RS:2;jenkins-hbase4:42285] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42285,1683197625528' 2023-05-04 10:53:46,923 DEBUG [RS:2;jenkins-hbase4:42285] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-05-04 10:53:46,923 DEBUG [RS:0;jenkins-hbase4:37343] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-05-04 10:53:46,923 DEBUG [RS:1;jenkins-hbase4:33581] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-05-04 10:53:46,923 DEBUG [RS:2;jenkins-hbase4:42285] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-05-04 10:53:46,924 DEBUG [RS:0;jenkins-hbase4:37343] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-05-04 10:53:46,924 INFO [RS:0;jenkins-hbase4:37343] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-05-04 10:53:46,924 DEBUG [RS:1;jenkins-hbase4:33581] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-05-04 10:53:46,924 INFO [RS:0;jenkins-hbase4:37343] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-05-04 10:53:46,924 INFO [RS:1;jenkins-hbase4:33581] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-05-04 10:53:46,926 INFO [RS:1;jenkins-hbase4:33581] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-05-04 10:53:46,925 DEBUG [RS:2;jenkins-hbase4:42285] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-05-04 10:53:46,926 INFO [RS:2;jenkins-hbase4:42285] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-05-04 10:53:46,926 INFO [RS:2;jenkins-hbase4:42285] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-05-04 10:53:46,956 DEBUG [jenkins-hbase4:40233] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-05-04 10:53:46,959 DEBUG [jenkins-hbase4:40233] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-04 10:53:46,965 DEBUG [jenkins-hbase4:40233] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-04 10:53:46,965 DEBUG [jenkins-hbase4:40233] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-04 10:53:46,965 DEBUG [jenkins-hbase4:40233] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-04 10:53:46,965 DEBUG [jenkins-hbase4:40233] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-04 10:53:46,968 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42285,1683197625528, state=OPENING 2023-05-04 10:53:46,976 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-05-04 10:53:46,977 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-04 10:53:46,978 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-05-04 10:53:46,982 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:53:47,041 INFO [RS:0;jenkins-hbase4:37343] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37343%2C1683197625438, suffix=, logDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/WALs/jenkins-hbase4.apache.org,37343,1683197625438, archiveDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/oldWALs, maxLogs=32 2023-05-04 10:53:47,041 INFO [RS:1;jenkins-hbase4:33581] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33581%2C1683197625489, suffix=, logDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/WALs/jenkins-hbase4.apache.org,33581,1683197625489, archiveDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/oldWALs, maxLogs=32 2023-05-04 10:53:47,041 INFO [RS:2;jenkins-hbase4:42285] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42285%2C1683197625528, suffix=, logDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/WALs/jenkins-hbase4.apache.org,42285,1683197625528, archiveDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/oldWALs, maxLogs=32 2023-05-04 10:53:47,063 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38321,DS-0b13d23a-a532-49b1-93c9-373b659adc8e,DISK] 2023-05-04 10:53:47,063 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44687,DS-ff8dcb33-2008-4522-bdfb-c059988b1bb3,DISK] 2023-05-04 10:53:47,063 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34481,DS-e2fcb6ee-ea17-48bd-ab8e-961f13d8341b,DISK] 2023-05-04 10:53:47,086 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38321,DS-0b13d23a-a532-49b1-93c9-373b659adc8e,DISK] 2023-05-04 10:53:47,088 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34481,DS-e2fcb6ee-ea17-48bd-ab8e-961f13d8341b,DISK] 2023-05-04 10:53:47,089 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34481,DS-e2fcb6ee-ea17-48bd-ab8e-961f13d8341b,DISK] 2023-05-04 10:53:47,089 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44687,DS-ff8dcb33-2008-4522-bdfb-c059988b1bb3,DISK] 2023-05-04 10:53:47,090 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44687,DS-ff8dcb33-2008-4522-bdfb-c059988b1bb3,DISK] 2023-05-04 10:53:47,091 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38321,DS-0b13d23a-a532-49b1-93c9-373b659adc8e,DISK] 2023-05-04 10:53:47,092 INFO [RS:0;jenkins-hbase4:37343] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/WALs/jenkins-hbase4.apache.org,37343,1683197625438/jenkins-hbase4.apache.org%2C37343%2C1683197625438.1683197627044 2023-05-04 10:53:47,097 DEBUG [RS:0;jenkins-hbase4:37343] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38321,DS-0b13d23a-a532-49b1-93c9-373b659adc8e,DISK], DatanodeInfoWithStorage[127.0.0.1:44687,DS-ff8dcb33-2008-4522-bdfb-c059988b1bb3,DISK], DatanodeInfoWithStorage[127.0.0.1:34481,DS-e2fcb6ee-ea17-48bd-ab8e-961f13d8341b,DISK]] 2023-05-04 10:53:47,098 INFO [RS:1;jenkins-hbase4:33581] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/WALs/jenkins-hbase4.apache.org,33581,1683197625489/jenkins-hbase4.apache.org%2C33581%2C1683197625489.1683197627045 2023-05-04 10:53:47,098 INFO [RS:2;jenkins-hbase4:42285] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/WALs/jenkins-hbase4.apache.org,42285,1683197625528/jenkins-hbase4.apache.org%2C42285%2C1683197625528.1683197627045 2023-05-04 10:53:47,098 DEBUG [RS:1;jenkins-hbase4:33581] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44687,DS-ff8dcb33-2008-4522-bdfb-c059988b1bb3,DISK], DatanodeInfoWithStorage[127.0.0.1:38321,DS-0b13d23a-a532-49b1-93c9-373b659adc8e,DISK], DatanodeInfoWithStorage[127.0.0.1:34481,DS-e2fcb6ee-ea17-48bd-ab8e-961f13d8341b,DISK]] 2023-05-04 10:53:47,099 DEBUG [RS:2;jenkins-hbase4:42285] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34481,DS-e2fcb6ee-ea17-48bd-ab8e-961f13d8341b,DISK], DatanodeInfoWithStorage[127.0.0.1:44687,DS-ff8dcb33-2008-4522-bdfb-c059988b1bb3,DISK], DatanodeInfoWithStorage[127.0.0.1:38321,DS-0b13d23a-a532-49b1-93c9-373b659adc8e,DISK]] 2023-05-04 10:53:47,166 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:47,168 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-05-04 10:53:47,171 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37000, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-05-04 10:53:47,186 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-05-04 10:53:47,186 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-04 10:53:47,189 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42285%2C1683197625528.meta, suffix=.meta, logDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/WALs/jenkins-hbase4.apache.org,42285,1683197625528, archiveDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/oldWALs, maxLogs=32 2023-05-04 10:53:47,209 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38321,DS-0b13d23a-a532-49b1-93c9-373b659adc8e,DISK] 2023-05-04 10:53:47,210 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44687,DS-ff8dcb33-2008-4522-bdfb-c059988b1bb3,DISK] 2023-05-04 10:53:47,211 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34481,DS-e2fcb6ee-ea17-48bd-ab8e-961f13d8341b,DISK] 2023-05-04 10:53:47,216 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/WALs/jenkins-hbase4.apache.org,42285,1683197625528/jenkins-hbase4.apache.org%2C42285%2C1683197625528.meta.1683197627191.meta 2023-05-04 10:53:47,216 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44687,DS-ff8dcb33-2008-4522-bdfb-c059988b1bb3,DISK], DatanodeInfoWithStorage[127.0.0.1:38321,DS-0b13d23a-a532-49b1-93c9-373b659adc8e,DISK], DatanodeInfoWithStorage[127.0.0.1:34481,DS-e2fcb6ee-ea17-48bd-ab8e-961f13d8341b,DISK]] 2023-05-04 10:53:47,217 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-05-04 10:53:47,223 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:53:47,224 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-05-04 10:53:47,240 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-05-04 10:53:47,242 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-05-04 10:53:47,247 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-05-04 10:53:47,247 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:53:47,247 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-05-04 10:53:47,247 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-05-04 10:53:47,251 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-05-04 10:53:47,252 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/info 2023-05-04 10:53:47,252 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/info 2023-05-04 10:53:47,253 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-05-04 10:53:47,254 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:47,254 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-05-04 10:53:47,255 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/rep_barrier 2023-05-04 10:53:47,255 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/rep_barrier 2023-05-04 10:53:47,256 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-05-04 10:53:47,257 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:47,257 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-05-04 10:53:47,258 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/table 2023-05-04 10:53:47,258 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/table 2023-05-04 10:53:47,259 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-05-04 10:53:47,259 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:47,261 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740 2023-05-04 10:53:47,264 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740 2023-05-04 10:53:47,268 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-05-04 10:53:47,270 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-05-04 10:53:47,271 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65566174, jitterRate=-0.02298787236213684}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-05-04 10:53:47,272 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-05-04 10:53:47,283 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1683197627158 2023-05-04 10:53:47,300 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-05-04 10:53:47,301 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-05-04 10:53:47,302 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42285,1683197625528, state=OPEN 2023-05-04 10:53:47,305 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-05-04 10:53:47,305 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-05-04 10:53:47,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-05-04 10:53:47,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42285,1683197625528 in 323 msec 2023-05-04 10:53:47,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-05-04 10:53:47,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 534 msec 2023-05-04 10:53:47,329 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 816 msec 2023-05-04 10:53:47,329 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1683197627329, completionTime=-1 2023-05-04 10:53:47,330 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-05-04 10:53:47,330 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:53:47,392 DEBUG [hconnection-0x451b64de-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-04 10:53:47,396 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37002, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-04 10:53:47,412 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:53:47,412 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197687412 2023-05-04 10:53:47,412 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197747412 2023-05-04 10:53:47,413 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 82 msec 2023-05-04 10:53:47,439 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40233,1683197624291-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:47,439 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40233,1683197624291-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:47,439 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40233,1683197624291-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:47,441 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:40233, period=300000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:47,441 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-05-04 10:53:47,452 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-05-04 10:53:47,460 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-05-04 10:53:47,461 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-05-04 10:53:47,471 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-05-04 10:53:47,473 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-05-04 10:53:47,476 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-04 10:53:47,498 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/hbase/namespace/468c84c2202e377e79810bff249e3ae7 2023-05-04 10:53:47,501 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/hbase/namespace/468c84c2202e377e79810bff249e3ae7 empty. 2023-05-04 10:53:47,501 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/hbase/namespace/468c84c2202e377e79810bff249e3ae7 2023-05-04 10:53:47,501 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-05-04 10:53:47,544 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-05-04 10:53:47,546 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 468c84c2202e377e79810bff249e3ae7, NAME => 'hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp 2023-05-04 10:53:47,571 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:53:47,571 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 468c84c2202e377e79810bff249e3ae7, disabling compactions & flushes 2023-05-04 10:53:47,571 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7. 2023-05-04 10:53:47,571 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7. 2023-05-04 10:53:47,571 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7. after waiting 0 ms 2023-05-04 10:53:47,572 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7. 2023-05-04 10:53:47,572 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7. 2023-05-04 10:53:47,572 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 468c84c2202e377e79810bff249e3ae7: 2023-05-04 10:53:47,576 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-05-04 10:53:47,591 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1683197627579"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197627579"}]},"ts":"1683197627579"} 2023-05-04 10:53:47,619 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-04 10:53:47,620 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-04 10:53:47,625 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197627621"}]},"ts":"1683197627621"} 2023-05-04 10:53:47,630 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-05-04 10:53:47,634 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-04 10:53:47,635 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-04 10:53:47,635 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-04 10:53:47,635 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-04 10:53:47,635 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-04 10:53:47,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=468c84c2202e377e79810bff249e3ae7, ASSIGN}] 2023-05-04 10:53:47,643 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=468c84c2202e377e79810bff249e3ae7, ASSIGN 2023-05-04 10:53:47,644 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=468c84c2202e377e79810bff249e3ae7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37343,1683197625438; forceNewPlan=false, retain=false 2023-05-04 10:53:47,797 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-04 10:53:47,798 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=468c84c2202e377e79810bff249e3ae7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:47,798 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1683197627798"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197627798"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197627798"}]},"ts":"1683197627798"} 2023-05-04 10:53:47,803 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 468c84c2202e377e79810bff249e3ae7, server=jenkins-hbase4.apache.org,37343,1683197625438}] 2023-05-04 10:53:47,957 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:47,958 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-05-04 10:53:47,964 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51304, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-05-04 10:53:47,973 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7. 2023-05-04 10:53:47,974 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 468c84c2202e377e79810bff249e3ae7, NAME => 'hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7.', STARTKEY => '', ENDKEY => ''} 2023-05-04 10:53:47,975 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:53:47,976 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 468c84c2202e377e79810bff249e3ae7 2023-05-04 10:53:47,976 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:53:47,976 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 468c84c2202e377e79810bff249e3ae7 2023-05-04 10:53:47,976 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 468c84c2202e377e79810bff249e3ae7 2023-05-04 10:53:47,979 INFO [StoreOpener-468c84c2202e377e79810bff249e3ae7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 468c84c2202e377e79810bff249e3ae7 2023-05-04 10:53:47,981 DEBUG [StoreOpener-468c84c2202e377e79810bff249e3ae7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/namespace/468c84c2202e377e79810bff249e3ae7/info 2023-05-04 10:53:47,981 DEBUG [StoreOpener-468c84c2202e377e79810bff249e3ae7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/namespace/468c84c2202e377e79810bff249e3ae7/info 2023-05-04 10:53:47,982 INFO [StoreOpener-468c84c2202e377e79810bff249e3ae7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 468c84c2202e377e79810bff249e3ae7 columnFamilyName info 2023-05-04 10:53:47,982 INFO [StoreOpener-468c84c2202e377e79810bff249e3ae7-1] regionserver.HStore(310): Store=468c84c2202e377e79810bff249e3ae7/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:47,984 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/namespace/468c84c2202e377e79810bff249e3ae7 2023-05-04 10:53:47,985 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/namespace/468c84c2202e377e79810bff249e3ae7 2023-05-04 10:53:47,989 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 468c84c2202e377e79810bff249e3ae7 2023-05-04 10:53:47,993 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/namespace/468c84c2202e377e79810bff249e3ae7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-04 10:53:47,994 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 468c84c2202e377e79810bff249e3ae7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74425564, jitterRate=0.10902732610702515}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-04 10:53:47,994 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 468c84c2202e377e79810bff249e3ae7: 2023-05-04 10:53:47,996 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7., pid=6, masterSystemTime=1683197627957 2023-05-04 10:53:48,001 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7. 2023-05-04 10:53:48,001 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7. 2023-05-04 10:53:48,003 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:48,003 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1683197628002"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197628002"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197628002"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197628002"}]},"ts":"1683197628002"} 2023-05-04 10:53:48,010 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-05-04 10:53:48,010 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 468c84c2202e377e79810bff249e3ae7, server=jenkins-hbase4.apache.org,37343,1683197625438 in 203 msec 2023-05-04 10:53:48,014 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-05-04 10:53:48,014 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=468c84c2202e377e79810bff249e3ae7, ASSIGN in 371 msec 2023-05-04 10:53:48,016 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-04 10:53:48,016 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197628016"}]},"ts":"1683197628016"} 2023-05-04 10:53:48,019 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-05-04 10:53:48,022 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-05-04 10:53:48,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 560 msec 2023-05-04 10:53:48,075 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-05-04 10:53:48,079 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-05-04 10:53:48,079 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-04 10:53:48,099 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-04 10:53:48,102 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51320, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-04 10:53:48,114 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-05-04 10:53:48,131 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-05-04 10:53:48,137 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 29 msec 2023-05-04 10:53:48,149 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:53:48,160 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-05-04 10:53:48,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 17 msec 2023-05-04 10:53:48,174 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-05-04 10:53:48,178 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-05-04 10:53:48,178 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.585sec 2023-05-04 10:53:48,181 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-05-04 10:53:48,183 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-05-04 10:53:48,183 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-05-04 10:53:48,184 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40233,1683197624291-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-05-04 10:53:48,185 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40233,1683197624291-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-05-04 10:53:48,193 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-05-04 10:53:48,261 DEBUG [Listener at localhost/35483] zookeeper.ReadOnlyZKClient(139): Connect 0x7b7d11f1 to 127.0.0.1:53687 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-04 10:53:48,266 DEBUG [Listener at localhost/35483] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@fdafbe7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-04 10:53:48,283 DEBUG [hconnection-0x2d8012cd-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-04 10:53:48,295 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37016, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-04 10:53:48,305 INFO [Listener at localhost/35483] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,40233,1683197624291 2023-05-04 10:53:48,343 INFO [Listener at localhost/35483] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=360, OpenFileDescriptor=697, MaxFileDescriptor=60000, SystemLoadAverage=374, ProcessCount=170, AvailableMemoryMB=6603 2023-05-04 10:53:48,348 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:53:48,357 DEBUG [Listener at localhost/35483] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-05-04 10:53:48,360 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59962, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-05-04 10:53:48,372 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-05-04 10:53:48,372 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-04 10:53:48,374 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-04 10:53:48,379 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,381 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-05-04 10:53:48,384 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-04 10:53:48,386 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:48,387 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433 empty. 2023-05-04 10:53:48,390 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:48,391 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-05-04 10:53:48,414 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-05-04 10:53:48,416 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => f9694df3407e6abdcc846e20c3b7c433, NAME => 'testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp 2023-05-04 10:53:48,437 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:53:48,437 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1604): Closing f9694df3407e6abdcc846e20c3b7c433, disabling compactions & flushes 2023-05-04 10:53:48,437 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:48,437 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:48,437 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. after waiting 0 ms 2023-05-04 10:53:48,437 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:48,438 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:48,438 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1558): Region close journal for f9694df3407e6abdcc846e20c3b7c433: 2023-05-04 10:53:48,442 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-05-04 10:53:48,444 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1683197628443"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197628443"}]},"ts":"1683197628443"} 2023-05-04 10:53:48,446 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-04 10:53:48,448 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-04 10:53:48,448 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197628448"}]},"ts":"1683197628448"} 2023-05-04 10:53:48,451 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-05-04 10:53:48,459 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-04 10:53:48,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-04 10:53:48,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-04 10:53:48,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-04 10:53:48,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-04 10:53:48,462 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f9694df3407e6abdcc846e20c3b7c433, ASSIGN}] 2023-05-04 10:53:48,465 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f9694df3407e6abdcc846e20c3b7c433, ASSIGN 2023-05-04 10:53:48,466 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f9694df3407e6abdcc846e20c3b7c433, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33581,1683197625489; forceNewPlan=false, retain=false 2023-05-04 10:53:48,617 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-04 10:53:48,618 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=f9694df3407e6abdcc846e20c3b7c433, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:48,618 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1683197628618"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197628618"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197628618"}]},"ts":"1683197628618"} 2023-05-04 10:53:48,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure f9694df3407e6abdcc846e20c3b7c433, server=jenkins-hbase4.apache.org,33581,1683197625489}] 2023-05-04 10:53:48,775 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:48,776 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-05-04 10:53:48,779 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40220, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-05-04 10:53:48,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:48,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f9694df3407e6abdcc846e20c3b7c433, NAME => 'testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433.', STARTKEY => '', ENDKEY => ''} 2023-05-04 10:53:48,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:53:48,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:48,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:53:48,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:48,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:48,789 INFO [StoreOpener-f9694df3407e6abdcc846e20c3b7c433-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:48,792 DEBUG [StoreOpener-f9694df3407e6abdcc846e20c3b7c433-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1 2023-05-04 10:53:48,792 DEBUG [StoreOpener-f9694df3407e6abdcc846e20c3b7c433-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1 2023-05-04 10:53:48,793 INFO [StoreOpener-f9694df3407e6abdcc846e20c3b7c433-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f9694df3407e6abdcc846e20c3b7c433 columnFamilyName cf1 2023-05-04 10:53:48,793 INFO [StoreOpener-f9694df3407e6abdcc846e20c3b7c433-1] regionserver.HStore(310): Store=f9694df3407e6abdcc846e20c3b7c433/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:48,794 INFO [StoreOpener-f9694df3407e6abdcc846e20c3b7c433-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:48,796 DEBUG [StoreOpener-f9694df3407e6abdcc846e20c3b7c433-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2 2023-05-04 10:53:48,796 DEBUG [StoreOpener-f9694df3407e6abdcc846e20c3b7c433-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2 2023-05-04 10:53:48,796 INFO [StoreOpener-f9694df3407e6abdcc846e20c3b7c433-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f9694df3407e6abdcc846e20c3b7c433 columnFamilyName cf2 2023-05-04 10:53:48,797 INFO [StoreOpener-f9694df3407e6abdcc846e20c3b7c433-1] regionserver.HStore(310): Store=f9694df3407e6abdcc846e20c3b7c433/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:48,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:48,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:48,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:53:48,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:48,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-04 10:53:48,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f9694df3407e6abdcc846e20c3b7c433; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74683658, jitterRate=0.11287322640419006}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:53:48,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f9694df3407e6abdcc846e20c3b7c433: 2023-05-04 10:53:48,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433., pid=11, masterSystemTime=1683197628775 2023-05-04 10:53:48,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:48,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:48,819 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=f9694df3407e6abdcc846e20c3b7c433, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:48,819 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1683197628818"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197628818"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197628818"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197628818"}]},"ts":"1683197628818"} 2023-05-04 10:53:48,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-05-04 10:53:48,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure f9694df3407e6abdcc846e20c3b7c433, server=jenkins-hbase4.apache.org,33581,1683197625489 in 201 msec 2023-05-04 10:53:48,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-05-04 10:53:48,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f9694df3407e6abdcc846e20c3b7c433, ASSIGN in 364 msec 2023-05-04 10:53:48,830 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-04 10:53:48,831 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197628830"}]},"ts":"1683197628830"} 2023-05-04 10:53:48,833 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-05-04 10:53:48,836 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-05-04 10:53:48,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 460 msec 2023-05-04 10:53:48,884 DEBUG [Listener at localhost/35483] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-04 10:53:48,888 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40236, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-04 10:53:48,909 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:53:48,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,916 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-04 10:53:48,916 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:48,917 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-04 10:53:48,917 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-04 10:53:48,918 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,918 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,919 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:48,919 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:48,919 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:48,919 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:48,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:48,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:48,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:48,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:48,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:48,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:48,920 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:48,920 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:53:48,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:48,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:48,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:48,921 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,921 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,921 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,923 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,923 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,923 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,923 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-04 10:53:48,923 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-04 10:53:48,924 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-04 10:53:48,924 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:48,924 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:48,924 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-04 10:53:48,925 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-04 10:53:48,924 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:48,925 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-04 10:53:48,925 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-04 10:53:48,925 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-04 10:53:48,925 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-04 10:53:48,925 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-04 10:53:48,926 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:53:48,926 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:53:48,926 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:48,926 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. started... 2023-05-04 10:53:48,928 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9694df3407e6abdcc846e20c3b7c433 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-04 10:53:48,928 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,928 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,929 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,929 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:48,929 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:48,929 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:49,027 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf1/9fabe844481c4ec896a0dfcfe65fdf6e 2023-05-04 10:53:49,097 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf2/c54ff23294824aff9d09dab20f3d85d2 2023-05-04 10:53:49,108 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf1/9fabe844481c4ec896a0dfcfe65fdf6e as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/9fabe844481c4ec896a0dfcfe65fdf6e 2023-05-04 10:53:49,119 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/9fabe844481c4ec896a0dfcfe65fdf6e, entries=2, sequenceid=6, filesize=4.8 K 2023-05-04 10:53:49,122 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf2/c54ff23294824aff9d09dab20f3d85d2 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/c54ff23294824aff9d09dab20f3d85d2 2023-05-04 10:53:49,131 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/c54ff23294824aff9d09dab20f3d85d2, entries=2, sequenceid=6, filesize=4.8 K 2023-05-04 10:53:49,133 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for f9694df3407e6abdcc846e20c3b7c433 in 205ms, sequenceid=6, compaction requested=false 2023-05-04 10:53:49,135 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-05-04 10:53:49,137 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9694df3407e6abdcc846e20c3b7c433: 2023-05-04 10:53:49,137 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:49,137 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:53:49,138 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:53:49,138 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,138 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-04 10:53:49,138 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-04 10:53:49,140 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,140 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:49,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:49,141 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,141 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:49,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:49,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:49,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:49,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-04 10:53:49,144 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-04 10:53:49,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@36bf503a[Count = 0] remaining members to acquire global barrier 2023-05-04 10:53:49,144 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,149 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,149 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,149 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,150 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,150 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:53:49,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,150 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-04 10:53:49,150 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-04 10:53:49,150 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-04 10:53:49,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,150 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:53:49,150 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-04 10:53:49,150 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-04 10:53:49,150 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:53:49,150 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-04 10:53:49,150 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:53:49,155 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-04 10:53:49,155 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-04 10:53:49,155 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:49,155 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,156 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-04 10:53:49,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:49,156 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-04 10:53:49,155 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:49,156 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:49,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:49,157 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-04 10:53:49,156 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-04 10:53:49,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:49,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:49,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:49,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,33581,1683197625489': 2023-05-04 10:53:49,162 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-04 10:53:49,162 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:53:49,162 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:53:49,162 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:53:49,163 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,163 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:53:49,165 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,165 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,165 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,165 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,165 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,165 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:49,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:49,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,165 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:49,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,166 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,168 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:49,169 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:49,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,172 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,173 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,173 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,185 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,185 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,185 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,185 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,185 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,185 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,185 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,186 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,186 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,186 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,186 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,186 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,186 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-04 10:53:49,186 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:53:49,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,186 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,186 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:49,186 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,187 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,188 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,188 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,189 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:53:49,189 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:53:49,290 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:53:49,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:53:49,320 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:53:49,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-04 10:53:49,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,324 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-04 10:53:49,325 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:49,326 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-04 10:53:49,326 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-04 10:53:49,326 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,326 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,330 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,330 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,330 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,330 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,330 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,331 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:53:49,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:49,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,331 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:49,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,332 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:49,332 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-04 10:53:49,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,332 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-04 10:53:49,332 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,332 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,332 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,332 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-04 10:53:49,332 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-04 10:53:49,332 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:49,332 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-04 10:53:49,333 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-04 10:53:49,333 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:49,332 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,334 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:53:49,334 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-04 10:53:49,334 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-04 10:53:49,334 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:49,334 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:53:49,334 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:49,335 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-04 10:53:49,335 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. started... 2023-05-04 10:53:49,334 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-04 10:53:49,335 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-04 10:53:49,335 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-04 10:53:49,335 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-04 10:53:49,335 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9694df3407e6abdcc846e20c3b7c433 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:53:49,338 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,338 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,338 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,338 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:49,339 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,339 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:49,360 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf1/f5aedde3be504243945a1517efb80f1e 2023-05-04 10:53:49,391 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf2/bcefe59a26404eee9e2d2c348951e36b 2023-05-04 10:53:49,402 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf1/f5aedde3be504243945a1517efb80f1e as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/f5aedde3be504243945a1517efb80f1e 2023-05-04 10:53:49,411 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/f5aedde3be504243945a1517efb80f1e, entries=10, sequenceid=19, filesize=5.0 K 2023-05-04 10:53:49,413 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf2/bcefe59a26404eee9e2d2c348951e36b as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/bcefe59a26404eee9e2d2c348951e36b 2023-05-04 10:53:49,421 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/bcefe59a26404eee9e2d2c348951e36b, entries=10, sequenceid=19, filesize=5.0 K 2023-05-04 10:53:49,423 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f9694df3407e6abdcc846e20c3b7c433 in 88ms, sequenceid=19, compaction requested=false 2023-05-04 10:53:49,423 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9694df3407e6abdcc846e20c3b7c433: 2023-05-04 10:53:49,423 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:49,423 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:53:49,424 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:53:49,424 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,424 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-04 10:53:49,424 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-04 10:53:49,426 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,426 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:49,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:49,427 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,427 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:49,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:49,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:49,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:49,429 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-04 10:53:49,429 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@525381c[Count = 0] remaining members to acquire global barrier 2023-05-04 10:53:49,429 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-04 10:53:49,429 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,432 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,432 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,432 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,432 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-04 10:53:49,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,432 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,433 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:53:49,432 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-04 10:53:49,432 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-04 10:53:49,433 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-04 10:53:49,433 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:53:49,433 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:53:49,433 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-04 10:53:49,433 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-04 10:53:49,433 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:53:49,435 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-04 10:53:49,435 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-04 10:53:49,435 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:49,435 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:49,436 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-04 10:53:49,436 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-04 10:53:49,437 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,437 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-04 10:53:49,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:49,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:49,437 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:49,437 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-04 10:53:49,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:49,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:49,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:49,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,33581,1683197625489': 2023-05-04 10:53:49,442 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:53:49,442 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-04 10:53:49,442 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:53:49,442 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:53:49,442 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,442 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:53:49,444 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,444 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,444 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,444 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,444 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:49,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:49,444 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,445 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,446 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:49,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,446 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,447 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:49,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:49,450 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,450 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,451 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,464 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,464 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,464 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,464 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,464 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,464 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,464 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,464 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,465 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-04 10:53:49,465 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:53:49,465 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,465 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:49,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,465 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,466 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,466 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,466 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,466 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,466 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,466 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,466 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,467 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:53:49,467 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:53:49,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,567 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:53:49,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:53:49,592 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:53:49,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-04 10:53:49,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,595 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-04 10:53:49,595 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:49,596 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-04 10:53:49,596 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-04 10:53:49,597 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,597 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,598 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,598 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,598 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,598 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,599 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,599 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:53:49,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,600 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,600 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,600 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:49,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:49,600 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:49,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,601 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,601 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,601 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-04 10:53:49,603 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,603 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-04 10:53:49,603 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:49,603 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-04 10:53:49,604 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-04 10:53:49,604 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,604 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-04 10:53:49,604 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-04 10:53:49,604 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,604 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:49,604 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-04 10:53:49,604 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-04 10:53:49,605 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:49,605 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-04 10:53:49,606 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-04 10:53:49,606 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-04 10:53:49,606 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-04 10:53:49,605 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-04 10:53:49,606 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:53:49,606 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:53:49,606 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:49,606 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. started... 2023-05-04 10:53:49,606 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9694df3407e6abdcc846e20c3b7c433 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:53:49,608 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,608 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,609 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,609 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:49,609 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,609 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:49,629 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf1/e3a41fb0f0014ea7a33d3097ef161d10 2023-05-04 10:53:49,663 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf2/3c0e08d76192474ea5f40c14ab9affd9 2023-05-04 10:53:49,674 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf1/e3a41fb0f0014ea7a33d3097ef161d10 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e3a41fb0f0014ea7a33d3097ef161d10 2023-05-04 10:53:49,684 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e3a41fb0f0014ea7a33d3097ef161d10, entries=10, sequenceid=32, filesize=5.0 K 2023-05-04 10:53:49,685 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf2/3c0e08d76192474ea5f40c14ab9affd9 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/3c0e08d76192474ea5f40c14ab9affd9 2023-05-04 10:53:49,694 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/3c0e08d76192474ea5f40c14ab9affd9, entries=10, sequenceid=32, filesize=5.0 K 2023-05-04 10:53:49,695 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f9694df3407e6abdcc846e20c3b7c433 in 89ms, sequenceid=32, compaction requested=true 2023-05-04 10:53:49,695 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9694df3407e6abdcc846e20c3b7c433: 2023-05-04 10:53:49,695 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:49,695 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:53:49,696 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:53:49,696 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,696 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-04 10:53:49,696 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-04 10:53:49,698 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,698 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:49,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:49,699 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,699 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:49,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:49,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:49,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:49,701 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-04 10:53:49,701 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5359fa88[Count = 0] remaining members to acquire global barrier 2023-05-04 10:53:49,702 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-04 10:53:49,702 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,704 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,704 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,704 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,704 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-04 10:53:49,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,704 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,704 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:53:49,704 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-04 10:53:49,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,704 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:53:49,705 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-04 10:53:49,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,705 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-04 10:53:49,705 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:53:49,705 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-04 10:53:49,705 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-04 10:53:49,705 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:53:49,709 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-04 10:53:49,709 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:49,709 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-04 10:53:49,710 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-04 10:53:49,710 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:49,710 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-04 10:53:49,711 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-04 10:53:49,712 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,712 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:49,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:49,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:49,712 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-04 10:53:49,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:49,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:49,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:49,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,33581,1683197625489': 2023-05-04 10:53:49,716 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:53:49,716 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-04 10:53:49,716 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:53:49,716 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:53:49,716 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,716 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:53:49,719 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,719 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,719 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:49,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:49,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,719 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,719 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,719 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:49,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,719 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,720 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,721 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:49,721 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:49,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,724 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,724 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,725 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,728 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,728 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,728 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,728 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,728 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,728 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,728 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,728 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,728 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:49,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,729 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-04 10:53:49,729 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:53:49,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,728 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,729 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,728 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,729 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:53:49,729 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,729 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,729 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:53:49,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,730 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,730 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,730 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,730 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,830 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:53:49,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:53:49,854 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:53:49,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-04 10:53:49,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,857 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-04 10:53:49,857 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:49,858 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-04 10:53:49,858 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-04 10:53:49,858 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,858 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,860 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,860 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,860 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,860 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,861 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,861 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:53:49,861 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,861 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,861 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:49,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:49,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:49,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,862 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,862 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,862 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,862 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-04 10:53:49,862 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-04 10:53:49,862 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,862 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-04 10:53:49,862 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,862 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,862 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-04 10:53:49,863 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-04 10:53:49,863 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:49,863 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-04 10:53:49,863 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-04 10:53:49,863 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:49,863 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-04 10:53:49,863 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:49,864 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-04 10:53:49,865 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-04 10:53:49,864 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-04 10:53:49,865 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-04 10:53:49,865 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-04 10:53:49,865 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:53:49,865 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:53:49,865 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:49,865 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. started... 2023-05-04 10:53:49,866 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9694df3407e6abdcc846e20c3b7c433 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:53:49,868 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,868 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,868 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,869 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:49,868 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,869 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:49,897 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf1/5311aa30a7524849b0616ac501c8533d 2023-05-04 10:53:49,927 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf2/1e9933b128f94382b2bb088b382daef8 2023-05-04 10:53:49,938 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf1/5311aa30a7524849b0616ac501c8533d as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/5311aa30a7524849b0616ac501c8533d 2023-05-04 10:53:49,946 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/5311aa30a7524849b0616ac501c8533d, entries=10, sequenceid=45, filesize=5.0 K 2023-05-04 10:53:49,947 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf2/1e9933b128f94382b2bb088b382daef8 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/1e9933b128f94382b2bb088b382daef8 2023-05-04 10:53:49,954 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/1e9933b128f94382b2bb088b382daef8, entries=10, sequenceid=45, filesize=5.0 K 2023-05-04 10:53:49,956 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f9694df3407e6abdcc846e20c3b7c433 in 90ms, sequenceid=45, compaction requested=true 2023-05-04 10:53:49,956 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9694df3407e6abdcc846e20c3b7c433: 2023-05-04 10:53:49,956 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:49,956 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:53:49,957 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:53:49,957 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,957 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-04 10:53:49,957 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-04 10:53:49,965 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,965 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:49,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:49,966 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,966 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:49,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:49,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:49,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:49,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-04 10:53:49,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31100b97[Count = 0] remaining members to acquire global barrier 2023-05-04 10:53:49,969 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-04 10:53:49,969 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,971 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,971 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:53:49,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,970 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-04 10:53:49,971 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-04 10:53:49,971 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-04 10:53:49,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,971 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:53:49,971 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-04 10:53:49,971 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:53:49,971 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-04 10:53:49,971 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-04 10:53:49,971 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:53:49,973 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-04 10:53:49,973 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:49,973 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-04 10:53:49,974 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-04 10:53:49,974 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-04 10:53:49,974 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:49,974 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,974 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-04 10:53:49,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:49,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:49,974 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:49,975 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-04 10:53:49,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:49,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:49,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:49,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,33581,1683197625489': 2023-05-04 10:53:49,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:53:49,979 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-04 10:53:49,980 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:53:49,980 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:53:49,980 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,980 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:53:49,982 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,982 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,982 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:49,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:49,983 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,983 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,983 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,983 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,984 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:49,984 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,985 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:49,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:49,988 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,988 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,988 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,992 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,992 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,992 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,992 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,992 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:49,992 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,992 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,992 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-04 10:53:49,993 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:53:49,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,992 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,992 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:49,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:49,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:49,993 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,993 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,993 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:49,993 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:53:49,993 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:49,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,994 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:53:49,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:49,994 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:49,994 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:49,994 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:49,994 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:49,994 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,094 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:53:50,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:53:50,119 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:53:50,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-04 10:53:50,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,121 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-04 10:53:50,121 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:50,122 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-04 10:53:50,122 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-04 10:53:50,123 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,123 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,124 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:50,124 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:50,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:50,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:50,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:50,124 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:50,124 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:50,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:50,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:50,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,125 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,125 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:53:50,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,125 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,125 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:50,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,126 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:50,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,126 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-04 10:53:50,126 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,126 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:50,126 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-04 10:53:50,126 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:50,126 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-04 10:53:50,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,127 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,127 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-04 10:53:50,127 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-04 10:53:50,127 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-04 10:53:50,128 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-04 10:53:50,127 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,128 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:50,128 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-04 10:53:50,128 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-04 10:53:50,128 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-04 10:53:50,129 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-04 10:53:50,130 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,130 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-04 10:53:50,130 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:50,131 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-04 10:53:50,132 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:53:50,132 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,132 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:50,132 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:53:50,132 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. started... 2023-05-04 10:53:50,132 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,132 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9694df3407e6abdcc846e20c3b7c433 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:53:50,132 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,132 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:50,132 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,132 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:50,150 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf1/e79cd78004fc45978b0103ca89fcc866 2023-05-04 10:53:50,178 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf2/f3e56502764349bf92bbf047c50d0130 2023-05-04 10:53:50,187 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf1/e79cd78004fc45978b0103ca89fcc866 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e79cd78004fc45978b0103ca89fcc866 2023-05-04 10:53:50,195 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e79cd78004fc45978b0103ca89fcc866, entries=10, sequenceid=58, filesize=5.0 K 2023-05-04 10:53:50,196 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf2/f3e56502764349bf92bbf047c50d0130 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/f3e56502764349bf92bbf047c50d0130 2023-05-04 10:53:50,204 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/f3e56502764349bf92bbf047c50d0130, entries=10, sequenceid=58, filesize=5.0 K 2023-05-04 10:53:50,205 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f9694df3407e6abdcc846e20c3b7c433 in 73ms, sequenceid=58, compaction requested=true 2023-05-04 10:53:50,206 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9694df3407e6abdcc846e20c3b7c433: 2023-05-04 10:53:50,206 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:50,206 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:53:50,208 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:53:50,208 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,209 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-04 10:53:50,209 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-04 10:53:50,210 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,210 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:50,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:50,211 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,211 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:50,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:50,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:50,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:50,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:50,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:50,214 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-04 10:53:50,214 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7de2b30f[Count = 0] remaining members to acquire global barrier 2023-05-04 10:53:50,214 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-04 10:53:50,214 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,215 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,215 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,215 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,215 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-04 10:53:50,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,216 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,216 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-04 10:53:50,216 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-04 10:53:50,216 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-04 10:53:50,216 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:53:50,216 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:53:50,216 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-04 10:53:50,216 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-04 10:53:50,216 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:53:50,216 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:53:50,217 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-04 10:53:50,218 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:50,218 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-04 10:53:50,219 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,219 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-04 10:53:50,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,219 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-04 10:53:50,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:50,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:50,219 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:50,219 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:50,220 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-04 10:53:50,220 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-04 10:53:50,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:50,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:50,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:50,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:50,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:50,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:50,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:50,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,33581,1683197625489': 2023-05-04 10:53:50,225 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:53:50,225 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-04 10:53:50,225 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:53:50,225 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:53:50,225 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,225 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:53:50,227 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,227 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,227 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:50,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:50,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:50,227 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,227 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:50,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,227 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:50,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,227 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:50,227 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:50,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:50,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:50,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:50,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:50,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:50,228 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:50,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:50,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,229 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:50,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:50,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:50,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:50,231 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,231 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:50,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,231 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:50,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:50,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:50,237 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:50,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:50,237 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:50,237 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:50,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:50,237 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:50,237 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:50,238 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:50,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:50,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:50,237 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:50,239 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:50,237 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:50,239 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:50,238 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:50,238 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-04 10:53:50,239 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:53:50,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:50,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:50,239 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:50,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:50,239 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:50,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:50,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:50,240 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,240 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:53:50,240 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:53:50,240 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:50,240 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,240 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,240 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,340 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:53:50,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:53:50,365 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:53:50,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-04 10:53:50,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,368 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-04 10:53:50,368 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:50,369 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-04 10:53:50,369 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-04 10:53:50,369 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,370 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,372 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,372 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:50,372 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:50,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:50,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:50,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:50,372 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,373 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:53:50,372 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:50,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:50,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:50,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:50,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,373 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,374 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:50,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,374 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,374 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:50,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,374 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,374 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-04 10:53:50,374 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,374 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-04 10:53:50,374 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-04 10:53:50,374 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:50,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:50,375 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-04 10:53:50,375 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-04 10:53:50,375 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-04 10:53:50,374 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,375 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-04 10:53:50,375 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,375 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:50,376 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-04 10:53:50,376 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-04 10:53:50,376 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,376 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-04 10:53:50,376 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:50,376 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:53:50,377 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:53:50,376 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. started... 2023-05-04 10:53:50,376 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:50,377 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-04 10:53:50,377 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f9694df3407e6abdcc846e20c3b7c433 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:53:50,377 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-04 10:53:50,377 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-04 10:53:50,377 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,378 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,378 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:50,379 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,380 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,380 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:50,397 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf1/07cac8d0bf404c40aec2ef4a7a7dc312 2023-05-04 10:53:50,422 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf2/d057826a49a1454283b664cb9256eb32 2023-05-04 10:53:50,430 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf1/07cac8d0bf404c40aec2ef4a7a7dc312 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/07cac8d0bf404c40aec2ef4a7a7dc312 2023-05-04 10:53:50,438 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/07cac8d0bf404c40aec2ef4a7a7dc312, entries=10, sequenceid=71, filesize=5.0 K 2023-05-04 10:53:50,439 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf2/d057826a49a1454283b664cb9256eb32 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/d057826a49a1454283b664cb9256eb32 2023-05-04 10:53:50,448 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/d057826a49a1454283b664cb9256eb32, entries=10, sequenceid=71, filesize=5.0 K 2023-05-04 10:53:50,449 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f9694df3407e6abdcc846e20c3b7c433 in 72ms, sequenceid=71, compaction requested=true 2023-05-04 10:53:50,449 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f9694df3407e6abdcc846e20c3b7c433: 2023-05-04 10:53:50,449 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:50,449 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:53:50,450 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:53:50,450 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,450 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-04 10:53:50,450 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-04 10:53:50,452 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,452 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:50,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:50,452 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,452 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:50,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:50,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:50,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:50,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:50,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:50,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-04 10:53:50,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@512b9177[Count = 0] remaining members to acquire global barrier 2023-05-04 10:53:50,455 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-04 10:53:50,455 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,457 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,457 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,457 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,457 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-04 10:53:50,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,457 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-04 10:53:50,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,457 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:53:50,457 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-04 10:53:50,457 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-04 10:53:50,457 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,457 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:53:50,458 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-04 10:53:50,457 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:53:50,458 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-04 10:53:50,458 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:53:50,459 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-04 10:53:50,459 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,459 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:50,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:50,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:50,459 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-04 10:53:50,461 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-04 10:53:50,461 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:50,461 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-04 10:53:50,462 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-04 10:53:50,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:50,462 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:50,462 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-04 10:53:50,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:50,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:50,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:50,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:50,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:50,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:50,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,33581,1683197625489': 2023-05-04 10:53:50,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:53:50,466 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-04 10:53:50,466 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:53:50,466 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:53:50,466 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,466 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:53:50,471 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,471 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:50,471 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:50,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:50,471 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,471 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,471 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:50,472 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,471 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:50,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:50,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,472 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:50,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:50,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:50,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:50,472 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:50,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:50,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:50,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:50,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:50,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:50,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:50,474 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:50,475 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:50,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,475 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:50,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:50,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:50,478 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:50,478 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:50,478 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:50,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:50,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:50,478 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:50,478 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:50,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:50,478 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:50,479 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:50,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:50,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:50,478 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:50,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:50,479 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:50,479 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-04 10:53:50,480 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:53:50,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:50,480 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:50,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:50,480 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:50,480 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:50,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:50,480 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:50,481 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:53:50,481 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,481 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:53:50,481 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:50,481 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,481 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,481 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-04 10:53:50,581 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:53:50,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:53:50,605 DEBUG [Listener at localhost/35483] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,612 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:50,613 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:50,613 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:50,613 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:50,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f9694df3407e6abdcc846e20c3b7c433, UNASSIGN}] 2023-05-04 10:53:50,624 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f9694df3407e6abdcc846e20c3b7c433, UNASSIGN 2023-05-04 10:53:50,625 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=f9694df3407e6abdcc846e20c3b7c433, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:50,626 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1683197630625"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197630625"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197630625"}]},"ts":"1683197630625"} 2023-05-04 10:53:50,628 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure f9694df3407e6abdcc846e20c3b7c433, server=jenkins-hbase4.apache.org,33581,1683197625489}] 2023-05-04 10:53:50,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:50,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f9694df3407e6abdcc846e20c3b7c433, disabling compactions & flushes 2023-05-04 10:53:50,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:50,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:50,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. after waiting 0 ms 2023-05-04 10:53:50,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:50,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing f9694df3407e6abdcc846e20c3b7c433 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-04 10:53:50,810 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf1/c2a3842a2e614387aab79778f08a6d5d 2023-05-04 10:53:50,834 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf2/6e17723095f64017bd55eeff862c239c 2023-05-04 10:53:50,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf1/c2a3842a2e614387aab79778f08a6d5d as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/c2a3842a2e614387aab79778f08a6d5d 2023-05-04 10:53:50,847 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/c2a3842a2e614387aab79778f08a6d5d, entries=8, sequenceid=82, filesize=5.0 K 2023-05-04 10:53:50,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.tmp/cf2/6e17723095f64017bd55eeff862c239c as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/6e17723095f64017bd55eeff862c239c 2023-05-04 10:53:50,857 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/6e17723095f64017bd55eeff862c239c, entries=8, sequenceid=82, filesize=5.0 K 2023-05-04 10:53:50,858 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for f9694df3407e6abdcc846e20c3b7c433 in 69ms, sequenceid=82, compaction requested=true 2023-05-04 10:53:50,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-04 10:53:50,880 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:53:50,881 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. 2023-05-04 10:53:50,881 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f9694df3407e6abdcc846e20c3b7c433: 2023-05-04 10:53:50,884 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:50,885 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=f9694df3407e6abdcc846e20c3b7c433, regionState=CLOSED 2023-05-04 10:53:50,886 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1683197630885"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197630885"}]},"ts":"1683197630885"} 2023-05-04 10:53:50,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-05-04 10:53:50,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure f9694df3407e6abdcc846e20c3b7c433, server=jenkins-hbase4.apache.org,33581,1683197625489 in 260 msec 2023-05-04 10:53:50,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-05-04 10:53:50,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f9694df3407e6abdcc846e20c3b7c433, UNASSIGN in 268 msec 2023-05-04 10:53:50,914 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=f9694df3407e6abdcc846e20c3b7c433, threads=14 2023-05-04 10:53:50,915 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/1e9933b128f94382b2bb088b382daef8 for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:50,915 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/3c0e08d76192474ea5f40c14ab9affd9 for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:50,915 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/6e17723095f64017bd55eeff862c239c for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:50,915 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/bcefe59a26404eee9e2d2c348951e36b for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:50,915 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/c54ff23294824aff9d09dab20f3d85d2 for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:50,916 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/d057826a49a1454283b664cb9256eb32 for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:50,916 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/f3e56502764349bf92bbf047c50d0130 for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:50,916 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/07cac8d0bf404c40aec2ef4a7a7dc312 for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:50,916 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/5311aa30a7524849b0616ac501c8533d for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:50,916 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/9fabe844481c4ec896a0dfcfe65fdf6e for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:50,917 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/c2a3842a2e614387aab79778f08a6d5d for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:50,917 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e3a41fb0f0014ea7a33d3097ef161d10 for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:50,917 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e79cd78004fc45978b0103ca89fcc866 for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:50,918 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/f5aedde3be504243945a1517efb80f1e for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:50,984 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/f3e56502764349bf92bbf047c50d0130, top=true 2023-05-04 10:53:50,997 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/6e17723095f64017bd55eeff862c239c, top=true 2023-05-04 10:53:51,011 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/c2a3842a2e614387aab79778f08a6d5d, top=true 2023-05-04 10:53:51,012 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e79cd78004fc45978b0103ca89fcc866, top=true 2023-05-04 10:53:51,038 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/d057826a49a1454283b664cb9256eb32, top=true 2023-05-04 10:53:51,041 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/07cac8d0bf404c40aec2ef4a7a7dc312, top=true 2023-05-04 10:53:51,056 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.splits/3c5731995a4a3f9265c35d3aa9f41462/cf2/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-6e17723095f64017bd55eeff862c239c for child: 3c5731995a4a3f9265c35d3aa9f41462, parent: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,056 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/6e17723095f64017bd55eeff862c239c for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,080 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.splits/3c5731995a4a3f9265c35d3aa9f41462/cf1/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-e79cd78004fc45978b0103ca89fcc866 for child: 3c5731995a4a3f9265c35d3aa9f41462, parent: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,080 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e79cd78004fc45978b0103ca89fcc866 for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,085 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.splits/3c5731995a4a3f9265c35d3aa9f41462/cf2/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-f3e56502764349bf92bbf047c50d0130 for child: 3c5731995a4a3f9265c35d3aa9f41462, parent: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,085 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/f3e56502764349bf92bbf047c50d0130 for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,111 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.splits/3c5731995a4a3f9265c35d3aa9f41462/cf1/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-07cac8d0bf404c40aec2ef4a7a7dc312 for child: 3c5731995a4a3f9265c35d3aa9f41462, parent: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,111 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/07cac8d0bf404c40aec2ef4a7a7dc312 for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,112 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.splits/3c5731995a4a3f9265c35d3aa9f41462/cf2/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-d057826a49a1454283b664cb9256eb32 for child: 3c5731995a4a3f9265c35d3aa9f41462, parent: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,112 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/d057826a49a1454283b664cb9256eb32 for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,125 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/.splits/3c5731995a4a3f9265c35d3aa9f41462/cf1/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-c2a3842a2e614387aab79778f08a6d5d for child: 3c5731995a4a3f9265c35d3aa9f41462, parent: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,125 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/c2a3842a2e614387aab79778f08a6d5d for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,157 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/bcefe59a26404eee9e2d2c348951e36b for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,200 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/9fabe844481c4ec896a0dfcfe65fdf6e for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,229 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/1e9933b128f94382b2bb088b382daef8 for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,234 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/5311aa30a7524849b0616ac501c8533d for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,236 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/3c0e08d76192474ea5f40c14ab9affd9 for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,238 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/c54ff23294824aff9d09dab20f3d85d2 for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,621 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/f5aedde3be504243945a1517efb80f1e for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,632 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e3a41fb0f0014ea7a33d3097ef161d10 for region: f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:51,633 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region f9694df3407e6abdcc846e20c3b7c433 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-05-04 10:53:51,688 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-04 10:53:51,690 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-04 10:53:51,693 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1683197631693"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1683197631693"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1683197631693"}]},"ts":"1683197631693"} 2023-05-04 10:53:51,693 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683197631693"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197631693"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197631693"}]},"ts":"1683197631693"} 2023-05-04 10:53:51,693 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683197631693"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197631693"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197631693"}]},"ts":"1683197631693"} 2023-05-04 10:53:51,742 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=068a8cdea69c89e269dfa78e5a33d56b, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=3c5731995a4a3f9265c35d3aa9f41462, ASSIGN}] 2023-05-04 10:53:51,744 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=3c5731995a4a3f9265c35d3aa9f41462, ASSIGN 2023-05-04 10:53:51,745 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=3c5731995a4a3f9265c35d3aa9f41462, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33581,1683197625489; forceNewPlan=false, retain=false 2023-05-04 10:53:51,746 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=068a8cdea69c89e269dfa78e5a33d56b, ASSIGN 2023-05-04 10:53:51,747 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=068a8cdea69c89e269dfa78e5a33d56b, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33581,1683197625489; forceNewPlan=false, retain=false 2023-05-04 10:53:51,895 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-04 10:53:51,897 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=3c5731995a4a3f9265c35d3aa9f41462, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:51,897 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683197631897"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197631897"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197631897"}]},"ts":"1683197631897"} 2023-05-04 10:53:51,900 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure 3c5731995a4a3f9265c35d3aa9f41462, server=jenkins-hbase4.apache.org,33581,1683197625489}] 2023-05-04 10:53:51,905 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=068a8cdea69c89e269dfa78e5a33d56b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:51,906 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683197631905"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197631905"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197631905"}]},"ts":"1683197631905"} 2023-05-04 10:53:51,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=15, state=RUNNABLE; OpenRegionProcedure 068a8cdea69c89e269dfa78e5a33d56b, server=jenkins-hbase4.apache.org,33581,1683197625489}] 2023-05-04 10:53:52,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462. 2023-05-04 10:53:52,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3c5731995a4a3f9265c35d3aa9f41462, NAME => 'testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.', STARTKEY => '26', ENDKEY => ''} 2023-05-04 10:53:52,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:53:52,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:52,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:53:52,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:52,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:52,061 INFO [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:52,062 DEBUG [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1 2023-05-04 10:53:52,063 DEBUG [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1 2023-05-04 10:53:52,063 INFO [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3c5731995a4a3f9265c35d3aa9f41462 columnFamilyName cf1 2023-05-04 10:53:52,102 DEBUG [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/5311aa30a7524849b0616ac501c8533d.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/5311aa30a7524849b0616ac501c8533d-top 2023-05-04 10:53:52,111 DEBUG [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/9fabe844481c4ec896a0dfcfe65fdf6e.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/9fabe844481c4ec896a0dfcfe65fdf6e-top 2023-05-04 10:53:52,118 DEBUG [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/e3a41fb0f0014ea7a33d3097ef161d10.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e3a41fb0f0014ea7a33d3097ef161d10-top 2023-05-04 10:53:52,124 DEBUG [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/f5aedde3be504243945a1517efb80f1e.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/f5aedde3be504243945a1517efb80f1e-top 2023-05-04 10:53:52,131 DEBUG [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-07cac8d0bf404c40aec2ef4a7a7dc312 2023-05-04 10:53:52,136 DEBUG [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-c2a3842a2e614387aab79778f08a6d5d 2023-05-04 10:53:52,141 DEBUG [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-e79cd78004fc45978b0103ca89fcc866 2023-05-04 10:53:52,142 INFO [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] regionserver.HStore(310): Store=3c5731995a4a3f9265c35d3aa9f41462/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:52,142 INFO [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:52,143 DEBUG [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2 2023-05-04 10:53:52,144 DEBUG [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2 2023-05-04 10:53:52,144 INFO [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3c5731995a4a3f9265c35d3aa9f41462 columnFamilyName cf2 2023-05-04 10:53:52,160 DEBUG [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/1e9933b128f94382b2bb088b382daef8.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/1e9933b128f94382b2bb088b382daef8-top 2023-05-04 10:53:52,166 DEBUG [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/3c0e08d76192474ea5f40c14ab9affd9.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/3c0e08d76192474ea5f40c14ab9affd9-top 2023-05-04 10:53:52,172 DEBUG [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/bcefe59a26404eee9e2d2c348951e36b.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/bcefe59a26404eee9e2d2c348951e36b-top 2023-05-04 10:53:52,177 DEBUG [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/c54ff23294824aff9d09dab20f3d85d2.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/c54ff23294824aff9d09dab20f3d85d2-top 2023-05-04 10:53:52,182 DEBUG [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-6e17723095f64017bd55eeff862c239c 2023-05-04 10:53:52,186 DEBUG [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-d057826a49a1454283b664cb9256eb32 2023-05-04 10:53:52,191 DEBUG [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-f3e56502764349bf92bbf047c50d0130 2023-05-04 10:53:52,191 INFO [StoreOpener-3c5731995a4a3f9265c35d3aa9f41462-1] regionserver.HStore(310): Store=3c5731995a4a3f9265c35d3aa9f41462/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:52,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:52,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:52,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:53:52,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:52,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3c5731995a4a3f9265c35d3aa9f41462; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71788174, jitterRate=0.06972715258598328}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:53:52,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3c5731995a4a3f9265c35d3aa9f41462: 2023-05-04 10:53:52,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462., pid=17, masterSystemTime=1683197632054 2023-05-04 10:53:52,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:53:52,206 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-04 10:53:52,206 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:53:52,206 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:53:52,209 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462. 2023-05-04 10:53:52,209 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HStore(1912): 3c5731995a4a3f9265c35d3aa9f41462/cf2 is initiating minor compaction (all files) 2023-05-04 10:53:52,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462. 2023-05-04 10:53:52,209 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 3c5731995a4a3f9265c35d3aa9f41462/cf2 in testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462. 2023-05-04 10:53:52,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462. 2023-05-04 10:53:52,209 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=3c5731995a4a3f9265c35d3aa9f41462, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:52,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b. 2023-05-04 10:53:52,209 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683197632209"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197632209"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197632209"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197632209"}]},"ts":"1683197632209"} 2023-05-04 10:53:52,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 068a8cdea69c89e269dfa78e5a33d56b, NAME => 'testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.', STARTKEY => '', ENDKEY => '26'} 2023-05-04 10:53:52,210 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/c54ff23294824aff9d09dab20f3d85d2.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/c54ff23294824aff9d09dab20f3d85d2-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/bcefe59a26404eee9e2d2c348951e36b.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/bcefe59a26404eee9e2d2c348951e36b-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/3c0e08d76192474ea5f40c14ab9affd9.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/3c0e08d76192474ea5f40c14ab9affd9-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/1e9933b128f94382b2bb088b382daef8.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/1e9933b128f94382b2bb088b382daef8-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-f3e56502764349bf92bbf047c50d0130, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-d057826a49a1454283b664cb9256eb32, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-6e17723095f64017bd55eeff862c239c] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/.tmp, totalSize=34.9 K 2023-05-04 10:53:52,211 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462. 2023-05-04 10:53:52,211 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HStore(1912): 3c5731995a4a3f9265c35d3aa9f41462/cf1 is initiating minor compaction (all files) 2023-05-04 10:53:52,211 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3c5731995a4a3f9265c35d3aa9f41462/cf1 in testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462. 2023-05-04 10:53:52,211 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/9fabe844481c4ec896a0dfcfe65fdf6e.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/9fabe844481c4ec896a0dfcfe65fdf6e-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/f5aedde3be504243945a1517efb80f1e.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/f5aedde3be504243945a1517efb80f1e-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/e3a41fb0f0014ea7a33d3097ef161d10.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e3a41fb0f0014ea7a33d3097ef161d10-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/5311aa30a7524849b0616ac501c8533d.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/5311aa30a7524849b0616ac501c8533d-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-e79cd78004fc45978b0103ca89fcc866, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-07cac8d0bf404c40aec2ef4a7a7dc312, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-c2a3842a2e614387aab79778f08a6d5d] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/.tmp, totalSize=34.9 K 2023-05-04 10:53:52,212 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting c54ff23294824aff9d09dab20f3d85d2.f9694df3407e6abdcc846e20c3b7c433, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683197628889 2023-05-04 10:53:52,213 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting 9fabe844481c4ec896a0dfcfe65fdf6e.f9694df3407e6abdcc846e20c3b7c433, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683197628889 2023-05-04 10:53:52,214 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting bcefe59a26404eee9e2d2c348951e36b.f9694df3407e6abdcc846e20c3b7c433, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683197629298 2023-05-04 10:53:52,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:53:52,215 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting 3c0e08d76192474ea5f40c14ab9affd9.f9694df3407e6abdcc846e20c3b7c433, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683197629570 2023-05-04 10:53:52,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 068a8cdea69c89e269dfa78e5a33d56b 2023-05-04 10:53:52,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:53:52,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 068a8cdea69c89e269dfa78e5a33d56b 2023-05-04 10:53:52,216 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting f5aedde3be504243945a1517efb80f1e.f9694df3407e6abdcc846e20c3b7c433, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683197629298 2023-05-04 10:53:52,216 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting 1e9933b128f94382b2bb088b382daef8.f9694df3407e6abdcc846e20c3b7c433, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683197629833 2023-05-04 10:53:52,217 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting e3a41fb0f0014ea7a33d3097ef161d10.f9694df3407e6abdcc846e20c3b7c433, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683197629570 2023-05-04 10:53:52,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 068a8cdea69c89e269dfa78e5a33d56b 2023-05-04 10:53:52,218 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-05-04 10:53:52,218 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure 3c5731995a4a3f9265c35d3aa9f41462, server=jenkins-hbase4.apache.org,33581,1683197625489 in 315 msec 2023-05-04 10:53:52,218 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-f3e56502764349bf92bbf047c50d0130, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683197630097 2023-05-04 10:53:52,218 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting 5311aa30a7524849b0616ac501c8533d.f9694df3407e6abdcc846e20c3b7c433, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683197629833 2023-05-04 10:53:52,219 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-e79cd78004fc45978b0103ca89fcc866, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683197630097 2023-05-04 10:53:52,219 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-d057826a49a1454283b664cb9256eb32, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683197630343 2023-05-04 10:53:52,220 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-6e17723095f64017bd55eeff862c239c, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683197630584 2023-05-04 10:53:52,220 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-07cac8d0bf404c40aec2ef4a7a7dc312, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683197630343 2023-05-04 10:53:52,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=3c5731995a4a3f9265c35d3aa9f41462, ASSIGN in 476 msec 2023-05-04 10:53:52,221 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-c2a3842a2e614387aab79778f08a6d5d, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683197630584 2023-05-04 10:53:52,223 INFO [StoreOpener-068a8cdea69c89e269dfa78e5a33d56b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 068a8cdea69c89e269dfa78e5a33d56b 2023-05-04 10:53:52,224 DEBUG [StoreOpener-068a8cdea69c89e269dfa78e5a33d56b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1 2023-05-04 10:53:52,225 DEBUG [StoreOpener-068a8cdea69c89e269dfa78e5a33d56b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1 2023-05-04 10:53:52,225 INFO [StoreOpener-068a8cdea69c89e269dfa78e5a33d56b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 068a8cdea69c89e269dfa78e5a33d56b columnFamilyName cf1 2023-05-04 10:53:52,245 DEBUG [StoreOpener-068a8cdea69c89e269dfa78e5a33d56b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/5311aa30a7524849b0616ac501c8533d.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/5311aa30a7524849b0616ac501c8533d-bottom 2023-05-04 10:53:52,264 DEBUG [StoreOpener-068a8cdea69c89e269dfa78e5a33d56b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/9fabe844481c4ec896a0dfcfe65fdf6e.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/9fabe844481c4ec896a0dfcfe65fdf6e-bottom 2023-05-04 10:53:52,265 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] throttle.PressureAwareThroughputController(145): 3c5731995a4a3f9265c35d3aa9f41462#cf2#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:53:52,287 DEBUG [StoreOpener-068a8cdea69c89e269dfa78e5a33d56b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/e3a41fb0f0014ea7a33d3097ef161d10.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e3a41fb0f0014ea7a33d3097ef161d10-bottom 2023-05-04 10:53:52,303 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3c5731995a4a3f9265c35d3aa9f41462#cf1#compaction#15 average throughput is 0.09 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:53:52,311 DEBUG [StoreOpener-068a8cdea69c89e269dfa78e5a33d56b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/f5aedde3be504243945a1517efb80f1e.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/f5aedde3be504243945a1517efb80f1e-bottom 2023-05-04 10:53:52,313 INFO [StoreOpener-068a8cdea69c89e269dfa78e5a33d56b-1] regionserver.HStore(310): Store=068a8cdea69c89e269dfa78e5a33d56b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:52,313 INFO [StoreOpener-068a8cdea69c89e269dfa78e5a33d56b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 068a8cdea69c89e269dfa78e5a33d56b 2023-05-04 10:53:52,315 DEBUG [StoreOpener-068a8cdea69c89e269dfa78e5a33d56b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2 2023-05-04 10:53:52,316 DEBUG [StoreOpener-068a8cdea69c89e269dfa78e5a33d56b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2 2023-05-04 10:53:52,316 INFO [StoreOpener-068a8cdea69c89e269dfa78e5a33d56b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 068a8cdea69c89e269dfa78e5a33d56b columnFamilyName cf2 2023-05-04 10:53:52,321 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/.tmp/cf2/778205de44164ac1af8a8182367dc8b7 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/778205de44164ac1af8a8182367dc8b7 2023-05-04 10:53:52,341 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3c5731995a4a3f9265c35d3aa9f41462/cf2 of 3c5731995a4a3f9265c35d3aa9f41462 into 778205de44164ac1af8a8182367dc8b7(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:53:52,341 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3c5731995a4a3f9265c35d3aa9f41462: 2023-05-04 10:53:52,341 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462., storeName=3c5731995a4a3f9265c35d3aa9f41462/cf2, priority=9, startTime=1683197632205; duration=0sec 2023-05-04 10:53:52,342 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:53:52,343 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/.tmp/cf1/64cbeffcbefc4fda9c4d61ecbf33c9bc as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/64cbeffcbefc4fda9c4d61ecbf33c9bc 2023-05-04 10:53:52,343 DEBUG [StoreOpener-068a8cdea69c89e269dfa78e5a33d56b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/1e9933b128f94382b2bb088b382daef8.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/1e9933b128f94382b2bb088b382daef8-bottom 2023-05-04 10:53:52,349 DEBUG [StoreOpener-068a8cdea69c89e269dfa78e5a33d56b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/3c0e08d76192474ea5f40c14ab9affd9.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/3c0e08d76192474ea5f40c14ab9affd9-bottom 2023-05-04 10:53:52,351 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3c5731995a4a3f9265c35d3aa9f41462/cf1 of 3c5731995a4a3f9265c35d3aa9f41462 into 64cbeffcbefc4fda9c4d61ecbf33c9bc(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:53:52,351 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3c5731995a4a3f9265c35d3aa9f41462: 2023-05-04 10:53:52,351 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462., storeName=3c5731995a4a3f9265c35d3aa9f41462/cf1, priority=9, startTime=1683197632204; duration=0sec 2023-05-04 10:53:52,352 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:53:52,355 DEBUG [StoreOpener-068a8cdea69c89e269dfa78e5a33d56b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/bcefe59a26404eee9e2d2c348951e36b.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/bcefe59a26404eee9e2d2c348951e36b-bottom 2023-05-04 10:53:52,361 DEBUG [StoreOpener-068a8cdea69c89e269dfa78e5a33d56b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/c54ff23294824aff9d09dab20f3d85d2.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/c54ff23294824aff9d09dab20f3d85d2-bottom 2023-05-04 10:53:52,361 INFO [StoreOpener-068a8cdea69c89e269dfa78e5a33d56b-1] regionserver.HStore(310): Store=068a8cdea69c89e269dfa78e5a33d56b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:52,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b 2023-05-04 10:53:52,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b 2023-05-04 10:53:52,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:53:52,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 068a8cdea69c89e269dfa78e5a33d56b 2023-05-04 10:53:52,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 068a8cdea69c89e269dfa78e5a33d56b; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67053309, jitterRate=-8.278340101242065E-4}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:53:52,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 068a8cdea69c89e269dfa78e5a33d56b: 2023-05-04 10:53:52,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b., pid=18, masterSystemTime=1683197632054 2023-05-04 10:53:52,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:53:52,371 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-05-04 10:53:52,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-04 10:53:52,371 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-05-04 10:53:52,372 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b. 2023-05-04 10:53:52,372 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b. 2023-05-04 10:53:52,372 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HStore(1912): 068a8cdea69c89e269dfa78e5a33d56b/cf1 is initiating minor compaction (all files) 2023-05-04 10:53:52,372 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HStore(1912): 068a8cdea69c89e269dfa78e5a33d56b/cf2 is initiating minor compaction (all files) 2023-05-04 10:53:52,372 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 068a8cdea69c89e269dfa78e5a33d56b/cf1 in testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b. 2023-05-04 10:53:52,372 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 068a8cdea69c89e269dfa78e5a33d56b/cf2 in testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b. 2023-05-04 10:53:52,373 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/9fabe844481c4ec896a0dfcfe65fdf6e.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/9fabe844481c4ec896a0dfcfe65fdf6e-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/f5aedde3be504243945a1517efb80f1e.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/f5aedde3be504243945a1517efb80f1e-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/e3a41fb0f0014ea7a33d3097ef161d10.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e3a41fb0f0014ea7a33d3097ef161d10-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/5311aa30a7524849b0616ac501c8533d.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/5311aa30a7524849b0616ac501c8533d-bottom] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/.tmp, totalSize=19.9 K 2023-05-04 10:53:52,373 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/c54ff23294824aff9d09dab20f3d85d2.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/c54ff23294824aff9d09dab20f3d85d2-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/bcefe59a26404eee9e2d2c348951e36b.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/bcefe59a26404eee9e2d2c348951e36b-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/3c0e08d76192474ea5f40c14ab9affd9.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/3c0e08d76192474ea5f40c14ab9affd9-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/1e9933b128f94382b2bb088b382daef8.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/1e9933b128f94382b2bb088b382daef8-bottom] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/.tmp, totalSize=19.9 K 2023-05-04 10:53:52,373 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting 9fabe844481c4ec896a0dfcfe65fdf6e.f9694df3407e6abdcc846e20c3b7c433, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683197628889 2023-05-04 10:53:52,374 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting c54ff23294824aff9d09dab20f3d85d2.f9694df3407e6abdcc846e20c3b7c433, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683197628889 2023-05-04 10:53:52,375 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting f5aedde3be504243945a1517efb80f1e.f9694df3407e6abdcc846e20c3b7c433, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683197629298 2023-05-04 10:53:52,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b. 2023-05-04 10:53:52,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b. 2023-05-04 10:53:52,376 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting bcefe59a26404eee9e2d2c348951e36b.f9694df3407e6abdcc846e20c3b7c433, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683197629298 2023-05-04 10:53:52,376 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting e3a41fb0f0014ea7a33d3097ef161d10.f9694df3407e6abdcc846e20c3b7c433, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683197629570 2023-05-04 10:53:52,376 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=068a8cdea69c89e269dfa78e5a33d56b, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:52,376 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting 3c0e08d76192474ea5f40c14ab9affd9.f9694df3407e6abdcc846e20c3b7c433, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683197629570 2023-05-04 10:53:52,377 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683197632376"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197632376"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197632376"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197632376"}]},"ts":"1683197632376"} 2023-05-04 10:53:52,377 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting 1e9933b128f94382b2bb088b382daef8.f9694df3407e6abdcc846e20c3b7c433, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683197629833 2023-05-04 10:53:52,377 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting 5311aa30a7524849b0616ac501c8533d.f9694df3407e6abdcc846e20c3b7c433, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683197629833 2023-05-04 10:53:52,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=15 2023-05-04 10:53:52,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=15, state=SUCCESS; OpenRegionProcedure 068a8cdea69c89e269dfa78e5a33d56b, server=jenkins-hbase4.apache.org,33581,1683197625489 in 471 msec 2023-05-04 10:53:52,386 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-05-04 10:53:52,386 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=068a8cdea69c89e269dfa78e5a33d56b, ASSIGN in 640 msec 2023-05-04 10:53:52,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 in 1.7810 sec 2023-05-04 10:53:52,392 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] throttle.PressureAwareThroughputController(145): 068a8cdea69c89e269dfa78e5a33d56b#cf2#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:53:52,394 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] throttle.PressureAwareThroughputController(145): 068a8cdea69c89e269dfa78e5a33d56b#cf1#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:53:52,412 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/.tmp/cf2/194604f6005246afab749c864146cd6c as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/194604f6005246afab749c864146cd6c 2023-05-04 10:53:52,413 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/.tmp/cf1/f19c594ea0644ca0ac3656cd2e16adb4 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/f19c594ea0644ca0ac3656cd2e16adb4 2023-05-04 10:53:52,420 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 068a8cdea69c89e269dfa78e5a33d56b/cf2 of 068a8cdea69c89e269dfa78e5a33d56b into 194604f6005246afab749c864146cd6c(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:53:52,421 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 068a8cdea69c89e269dfa78e5a33d56b: 2023-05-04 10:53:52,421 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 068a8cdea69c89e269dfa78e5a33d56b/cf1 of 068a8cdea69c89e269dfa78e5a33d56b into f19c594ea0644ca0ac3656cd2e16adb4(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:53:52,421 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b., storeName=068a8cdea69c89e269dfa78e5a33d56b/cf2, priority=12, startTime=1683197632371; duration=0sec 2023-05-04 10:53:52,423 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 068a8cdea69c89e269dfa78e5a33d56b: 2023-05-04 10:53:52,423 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b., storeName=068a8cdea69c89e269dfa78e5a33d56b/cf1, priority=12, startTime=1683197632371; duration=0sec 2023-05-04 10:53:52,423 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:53:52,423 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:53:52,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33581] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:40236 deadline: 1683197692624, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433. is not online on jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:52,760 DEBUG [Listener at localhost/35483] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-05-04 10:53:52,772 INFO [Listener at localhost/35483] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-05-04 10:53:52,774 DEBUG [Listener at localhost/35483] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-05-04 10:53:52,805 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-04 10:53:52,813 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:53:52,828 INFO [Listener at localhost/35483] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-05-04 10:53:52,838 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-05-04 10:53:52,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:52,855 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197632855"}]},"ts":"1683197632855"} 2023-05-04 10:53:52,858 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-05-04 10:53:52,860 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-05-04 10:53:52,865 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=068a8cdea69c89e269dfa78e5a33d56b, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=3c5731995a4a3f9265c35d3aa9f41462, UNASSIGN}] 2023-05-04 10:53:52,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-05-04 10:53:52,866 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=3c5731995a4a3f9265c35d3aa9f41462, UNASSIGN 2023-05-04 10:53:52,870 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=3c5731995a4a3f9265c35d3aa9f41462, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:52,871 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683197632868"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197632868"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197632868"}]},"ts":"1683197632868"} 2023-05-04 10:53:52,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 3c5731995a4a3f9265c35d3aa9f41462, server=jenkins-hbase4.apache.org,33581,1683197625489}] 2023-05-04 10:53:52,881 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=068a8cdea69c89e269dfa78e5a33d56b, UNASSIGN 2023-05-04 10:53:52,882 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=068a8cdea69c89e269dfa78e5a33d56b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:52,882 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683197632882"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197632882"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197632882"}]},"ts":"1683197632882"} 2023-05-04 10:53:52,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 068a8cdea69c89e269dfa78e5a33d56b, server=jenkins-hbase4.apache.org,33581,1683197625489}] 2023-05-04 10:53:52,902 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-05-04 10:53:52,903 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-05-04 10:53:52,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-05-04 10:53:53,029 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 068a8cdea69c89e269dfa78e5a33d56b 2023-05-04 10:53:53,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 068a8cdea69c89e269dfa78e5a33d56b, disabling compactions & flushes 2023-05-04 10:53:53,029 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b. 2023-05-04 10:53:53,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b. 2023-05-04 10:53:53,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b. after waiting 0 ms 2023-05-04 10:53:53,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b. 2023-05-04 10:53:53,035 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/9fabe844481c4ec896a0dfcfe65fdf6e.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/9fabe844481c4ec896a0dfcfe65fdf6e-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/f5aedde3be504243945a1517efb80f1e.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/f5aedde3be504243945a1517efb80f1e-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/e3a41fb0f0014ea7a33d3097ef161d10.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e3a41fb0f0014ea7a33d3097ef161d10-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/5311aa30a7524849b0616ac501c8533d.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/5311aa30a7524849b0616ac501c8533d-bottom] to archive 2023-05-04 10:53:53,037 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:53:53,044 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/9fabe844481c4ec896a0dfcfe65fdf6e.f9694df3407e6abdcc846e20c3b7c433 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/9fabe844481c4ec896a0dfcfe65fdf6e.f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:53,046 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/f5aedde3be504243945a1517efb80f1e.f9694df3407e6abdcc846e20c3b7c433 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/f5aedde3be504243945a1517efb80f1e.f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:53,047 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/e3a41fb0f0014ea7a33d3097ef161d10.f9694df3407e6abdcc846e20c3b7c433 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/e3a41fb0f0014ea7a33d3097ef161d10.f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:53,049 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/5311aa30a7524849b0616ac501c8533d.f9694df3407e6abdcc846e20c3b7c433 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/5311aa30a7524849b0616ac501c8533d.f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:53,076 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/c54ff23294824aff9d09dab20f3d85d2.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/c54ff23294824aff9d09dab20f3d85d2-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/bcefe59a26404eee9e2d2c348951e36b.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/bcefe59a26404eee9e2d2c348951e36b-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/3c0e08d76192474ea5f40c14ab9affd9.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/3c0e08d76192474ea5f40c14ab9affd9-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/1e9933b128f94382b2bb088b382daef8.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/1e9933b128f94382b2bb088b382daef8-bottom] to archive 2023-05-04 10:53:53,077 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:53:53,079 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/c54ff23294824aff9d09dab20f3d85d2.f9694df3407e6abdcc846e20c3b7c433 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/c54ff23294824aff9d09dab20f3d85d2.f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:53,081 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/bcefe59a26404eee9e2d2c348951e36b.f9694df3407e6abdcc846e20c3b7c433 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/bcefe59a26404eee9e2d2c348951e36b.f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:53,082 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/3c0e08d76192474ea5f40c14ab9affd9.f9694df3407e6abdcc846e20c3b7c433 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/3c0e08d76192474ea5f40c14ab9affd9.f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:53,084 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/1e9933b128f94382b2bb088b382daef8.f9694df3407e6abdcc846e20c3b7c433 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/1e9933b128f94382b2bb088b382daef8.f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:53,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-04 10:53:53,089 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:53:53,090 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b. 2023-05-04 10:53:53,090 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 068a8cdea69c89e269dfa78e5a33d56b: 2023-05-04 10:53:53,092 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 068a8cdea69c89e269dfa78e5a33d56b 2023-05-04 10:53:53,092 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:53,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3c5731995a4a3f9265c35d3aa9f41462, disabling compactions & flushes 2023-05-04 10:53:53,092 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462. 2023-05-04 10:53:53,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462. 2023-05-04 10:53:53,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462. after waiting 0 ms 2023-05-04 10:53:53,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462. 2023-05-04 10:53:53,093 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=068a8cdea69c89e269dfa78e5a33d56b, regionState=CLOSED 2023-05-04 10:53:53,093 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683197633093"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197633093"}]},"ts":"1683197633093"} 2023-05-04 10:53:53,094 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/9fabe844481c4ec896a0dfcfe65fdf6e.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/9fabe844481c4ec896a0dfcfe65fdf6e-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/f5aedde3be504243945a1517efb80f1e.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/f5aedde3be504243945a1517efb80f1e-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/e3a41fb0f0014ea7a33d3097ef161d10.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e3a41fb0f0014ea7a33d3097ef161d10-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/5311aa30a7524849b0616ac501c8533d.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/5311aa30a7524849b0616ac501c8533d-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-e79cd78004fc45978b0103ca89fcc866, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-07cac8d0bf404c40aec2ef4a7a7dc312, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-c2a3842a2e614387aab79778f08a6d5d] to archive 2023-05-04 10:53:53,095 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:53:53,099 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/9fabe844481c4ec896a0dfcfe65fdf6e.f9694df3407e6abdcc846e20c3b7c433 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/9fabe844481c4ec896a0dfcfe65fdf6e.f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:53,099 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-05-04 10:53:53,099 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 068a8cdea69c89e269dfa78e5a33d56b, server=jenkins-hbase4.apache.org,33581,1683197625489 in 209 msec 2023-05-04 10:53:53,101 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/f5aedde3be504243945a1517efb80f1e.f9694df3407e6abdcc846e20c3b7c433 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/f5aedde3be504243945a1517efb80f1e.f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:53,101 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=068a8cdea69c89e269dfa78e5a33d56b, UNASSIGN in 234 msec 2023-05-04 10:53:53,102 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/e3a41fb0f0014ea7a33d3097ef161d10.f9694df3407e6abdcc846e20c3b7c433 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/e3a41fb0f0014ea7a33d3097ef161d10.f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:53,104 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/5311aa30a7524849b0616ac501c8533d.f9694df3407e6abdcc846e20c3b7c433 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/5311aa30a7524849b0616ac501c8533d.f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:53,105 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-e79cd78004fc45978b0103ca89fcc866 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-e79cd78004fc45978b0103ca89fcc866 2023-05-04 10:53:53,107 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-07cac8d0bf404c40aec2ef4a7a7dc312 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-07cac8d0bf404c40aec2ef4a7a7dc312 2023-05-04 10:53:53,108 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-c2a3842a2e614387aab79778f08a6d5d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-c2a3842a2e614387aab79778f08a6d5d 2023-05-04 10:53:53,110 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/c54ff23294824aff9d09dab20f3d85d2.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/c54ff23294824aff9d09dab20f3d85d2-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/bcefe59a26404eee9e2d2c348951e36b.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/bcefe59a26404eee9e2d2c348951e36b-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/3c0e08d76192474ea5f40c14ab9affd9.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/3c0e08d76192474ea5f40c14ab9affd9-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/1e9933b128f94382b2bb088b382daef8.f9694df3407e6abdcc846e20c3b7c433->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/1e9933b128f94382b2bb088b382daef8-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-f3e56502764349bf92bbf047c50d0130, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-d057826a49a1454283b664cb9256eb32, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-6e17723095f64017bd55eeff862c239c] to archive 2023-05-04 10:53:53,111 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:53:53,113 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/c54ff23294824aff9d09dab20f3d85d2.f9694df3407e6abdcc846e20c3b7c433 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/c54ff23294824aff9d09dab20f3d85d2.f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:53,114 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/bcefe59a26404eee9e2d2c348951e36b.f9694df3407e6abdcc846e20c3b7c433 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/bcefe59a26404eee9e2d2c348951e36b.f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:53,115 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/3c0e08d76192474ea5f40c14ab9affd9.f9694df3407e6abdcc846e20c3b7c433 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/3c0e08d76192474ea5f40c14ab9affd9.f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:53,117 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/1e9933b128f94382b2bb088b382daef8.f9694df3407e6abdcc846e20c3b7c433 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/1e9933b128f94382b2bb088b382daef8.f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:53,118 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-f3e56502764349bf92bbf047c50d0130 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-f3e56502764349bf92bbf047c50d0130 2023-05-04 10:53:53,119 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-d057826a49a1454283b664cb9256eb32 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-d057826a49a1454283b664cb9256eb32 2023-05-04 10:53:53,121 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-6e17723095f64017bd55eeff862c239c to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/testSplitTableRegionUnevenDaughter=f9694df3407e6abdcc846e20c3b7c433-6e17723095f64017bd55eeff862c239c 2023-05-04 10:53:53,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-04 10:53:53,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:53:53,127 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462. 2023-05-04 10:53:53,127 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3c5731995a4a3f9265c35d3aa9f41462: 2023-05-04 10:53:53,129 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:53,130 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=3c5731995a4a3f9265c35d3aa9f41462, regionState=CLOSED 2023-05-04 10:53:53,130 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683197633130"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197633130"}]},"ts":"1683197633130"} 2023-05-04 10:53:53,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-05-04 10:53:53,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 3c5731995a4a3f9265c35d3aa9f41462, server=jenkins-hbase4.apache.org,33581,1683197625489 in 259 msec 2023-05-04 10:53:53,137 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-05-04 10:53:53,138 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=3c5731995a4a3f9265c35d3aa9f41462, UNASSIGN in 270 msec 2023-05-04 10:53:53,139 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197633138"}]},"ts":"1683197633138"} 2023-05-04 10:53:53,140 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-05-04 10:53:53,142 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-05-04 10:53:53,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 302 msec 2023-05-04 10:53:53,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-05-04 10:53:53,176 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-05-04 10:53:53,182 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-05-04 10:53:53,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:53,191 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:53,193 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:53,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-05-04 10:53:53,200 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:53,201 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b 2023-05-04 10:53:53,201 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:53,204 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/recovered.edits] 2023-05-04 10:53:53,204 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/recovered.edits] 2023-05-04 10:53:53,204 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/recovered.edits] 2023-05-04 10:53:53,209 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/64cbeffcbefc4fda9c4d61ecbf33c9bc to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf1/64cbeffcbefc4fda9c4d61ecbf33c9bc 2023-05-04 10:53:53,209 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/f19c594ea0644ca0ac3656cd2e16adb4 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf1/f19c594ea0644ca0ac3656cd2e16adb4 2023-05-04 10:53:53,209 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/07cac8d0bf404c40aec2ef4a7a7dc312 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/07cac8d0bf404c40aec2ef4a7a7dc312 2023-05-04 10:53:53,212 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/5311aa30a7524849b0616ac501c8533d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/5311aa30a7524849b0616ac501c8533d 2023-05-04 10:53:53,213 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/194604f6005246afab749c864146cd6c to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/cf2/194604f6005246afab749c864146cd6c 2023-05-04 10:53:53,213 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/778205de44164ac1af8a8182367dc8b7 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/cf2/778205de44164ac1af8a8182367dc8b7 2023-05-04 10:53:53,213 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/9fabe844481c4ec896a0dfcfe65fdf6e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/9fabe844481c4ec896a0dfcfe65fdf6e 2023-05-04 10:53:53,215 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/c2a3842a2e614387aab79778f08a6d5d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/c2a3842a2e614387aab79778f08a6d5d 2023-05-04 10:53:53,217 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e3a41fb0f0014ea7a33d3097ef161d10 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e3a41fb0f0014ea7a33d3097ef161d10 2023-05-04 10:53:53,217 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/recovered.edits/90.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b/recovered.edits/90.seqid 2023-05-04 10:53:53,218 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/recovered.edits/90.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462/recovered.edits/90.seqid 2023-05-04 10:53:53,218 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/068a8cdea69c89e269dfa78e5a33d56b 2023-05-04 10:53:53,219 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:53,219 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e79cd78004fc45978b0103ca89fcc866 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/e79cd78004fc45978b0103ca89fcc866 2023-05-04 10:53:53,220 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/f5aedde3be504243945a1517efb80f1e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf1/f5aedde3be504243945a1517efb80f1e 2023-05-04 10:53:53,223 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/1e9933b128f94382b2bb088b382daef8 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/1e9933b128f94382b2bb088b382daef8 2023-05-04 10:53:53,225 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/3c0e08d76192474ea5f40c14ab9affd9 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/3c0e08d76192474ea5f40c14ab9affd9 2023-05-04 10:53:53,226 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/6e17723095f64017bd55eeff862c239c to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/6e17723095f64017bd55eeff862c239c 2023-05-04 10:53:53,228 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/bcefe59a26404eee9e2d2c348951e36b to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/bcefe59a26404eee9e2d2c348951e36b 2023-05-04 10:53:53,229 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/c54ff23294824aff9d09dab20f3d85d2 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/c54ff23294824aff9d09dab20f3d85d2 2023-05-04 10:53:53,231 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/d057826a49a1454283b664cb9256eb32 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/d057826a49a1454283b664cb9256eb32 2023-05-04 10:53:53,233 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/f3e56502764349bf92bbf047c50d0130 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/cf2/f3e56502764349bf92bbf047c50d0130 2023-05-04 10:53:53,236 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/recovered.edits/85.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433/recovered.edits/85.seqid 2023-05-04 10:53:53,237 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionUnevenDaughter/f9694df3407e6abdcc846e20c3b7c433 2023-05-04 10:53:53,237 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-05-04 10:53:53,240 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:53,261 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-05-04 10:53:53,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-05-04 10:53:53,304 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-05-04 10:53:53,306 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:53,306 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-05-04 10:53:53,306 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197633306"}]},"ts":"9223372036854775807"} 2023-05-04 10:53:53,306 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197633306"}]},"ts":"9223372036854775807"} 2023-05-04 10:53:53,306 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197633306"}]},"ts":"9223372036854775807"} 2023-05-04 10:53:53,310 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-04 10:53:53,310 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => f9694df3407e6abdcc846e20c3b7c433, NAME => 'testSplitTableRegionUnevenDaughter,,1683197628377.f9694df3407e6abdcc846e20c3b7c433.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 068a8cdea69c89e269dfa78e5a33d56b, NAME => 'testSplitTableRegionUnevenDaughter,,1683197630605.068a8cdea69c89e269dfa78e5a33d56b.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 3c5731995a4a3f9265c35d3aa9f41462, NAME => 'testSplitTableRegionUnevenDaughter,26,1683197630605.3c5731995a4a3f9265c35d3aa9f41462.', STARTKEY => '26', ENDKEY => ''}] 2023-05-04 10:53:53,310 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-05-04 10:53:53,310 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683197633310"}]},"ts":"9223372036854775807"} 2023-05-04 10:53:53,312 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-05-04 10:53:53,314 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:53,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 131 msec 2023-05-04 10:53:53,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-05-04 10:53:53,497 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-05-04 10:53:53,515 INFO [Listener at localhost/35483] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=393 (was 360) Potentially hanging thread: hconnection-0x451b64de-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x451b64de-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1792210751_15 at /127.0.0.1:47172 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1792210751_15 at /127.0.0.1:54764 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1792210751_15 at /127.0.0.1:54836 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1792210751_15 at /127.0.0.1:54832 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1792210751_15 at /127.0.0.1:33222 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1792210751_15 at /127.0.0.1:54696 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1792210751_15 at /127.0.0.1:55016 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1792210751_15 at /127.0.0.1:54802 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1792210751_15 at /127.0.0.1:46944 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1792210751_15 at /127.0.0.1:47112 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x451b64de-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x451b64de-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1792210751_15 at /127.0.0.1:55000 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: (jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1792210751_15 at /127.0.0.1:33290 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x451b64de-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1792210751_15 at /127.0.0.1:54752 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1792210751_15 at /127.0.0.1:54778 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1792210751_15 at /127.0.0.1:47410 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:33581-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1792210751_15 at /127.0.0.1:33304 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=846 (was 697) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=384 (was 374) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=6537 (was 6603) 2023-05-04 10:53:53,530 INFO [Listener at localhost/35483] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=393, OpenFileDescriptor=846, MaxFileDescriptor=60000, SystemLoadAverage=384, ProcessCount=170, AvailableMemoryMB=6536 2023-05-04 10:53:53,530 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:53:53,536 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-04 10:53:53,537 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-04 10:53:53,539 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:53:53,541 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-05-04 10:53:53,542 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-04 10:53:53,545 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:53,545 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e empty. 2023-05-04 10:53:53,546 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:53,546 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-05-04 10:53:53,558 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-05-04 10:53:53,559 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7675): creating {ENCODED => fdafca090ed7cc34c4ad878aac762e9e, NAME => 'testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp 2023-05-04 10:53:53,569 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:53:53,569 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1604): Closing fdafca090ed7cc34c4ad878aac762e9e, disabling compactions & flushes 2023-05-04 10:53:53,569 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:53,569 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:53,569 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. after waiting 0 ms 2023-05-04 10:53:53,569 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:53,569 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:53,569 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1558): Region close journal for fdafca090ed7cc34c4ad878aac762e9e: 2023-05-04 10:53:53,572 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-05-04 10:53:53,573 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683197633573"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197633573"}]},"ts":"1683197633573"} 2023-05-04 10:53:53,575 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-04 10:53:53,575 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-04 10:53:53,576 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197633576"}]},"ts":"1683197633576"} 2023-05-04 10:53:53,577 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-05-04 10:53:53,580 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-04 10:53:53,581 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-04 10:53:53,581 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-04 10:53:53,581 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-04 10:53:53,581 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-04 10:53:53,581 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, ASSIGN}] 2023-05-04 10:53:53,583 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, ASSIGN 2023-05-04 10:53:53,584 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42285,1683197625528; forceNewPlan=false, retain=false 2023-05-04 10:53:53,734 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-04 10:53:53,736 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=fdafca090ed7cc34c4ad878aac762e9e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:53,736 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683197633736"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197633736"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197633736"}]},"ts":"1683197633736"} 2023-05-04 10:53:53,739 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:53:53,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:53,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fdafca090ed7cc34c4ad878aac762e9e, NAME => 'testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e.', STARTKEY => '', ENDKEY => ''} 2023-05-04 10:53:53,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:53:53,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:53,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:53:53,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:53,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:53,898 INFO [StoreOpener-fdafca090ed7cc34c4ad878aac762e9e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:53,899 DEBUG [StoreOpener-fdafca090ed7cc34c4ad878aac762e9e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1 2023-05-04 10:53:53,899 DEBUG [StoreOpener-fdafca090ed7cc34c4ad878aac762e9e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1 2023-05-04 10:53:53,900 INFO [StoreOpener-fdafca090ed7cc34c4ad878aac762e9e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fdafca090ed7cc34c4ad878aac762e9e columnFamilyName cf1 2023-05-04 10:53:53,900 INFO [StoreOpener-fdafca090ed7cc34c4ad878aac762e9e-1] regionserver.HStore(310): Store=fdafca090ed7cc34c4ad878aac762e9e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:53,900 INFO [StoreOpener-fdafca090ed7cc34c4ad878aac762e9e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:53,902 DEBUG [StoreOpener-fdafca090ed7cc34c4ad878aac762e9e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2 2023-05-04 10:53:53,902 DEBUG [StoreOpener-fdafca090ed7cc34c4ad878aac762e9e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2 2023-05-04 10:53:53,903 INFO [StoreOpener-fdafca090ed7cc34c4ad878aac762e9e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fdafca090ed7cc34c4ad878aac762e9e columnFamilyName cf2 2023-05-04 10:53:53,904 INFO [StoreOpener-fdafca090ed7cc34c4ad878aac762e9e-1] regionserver.HStore(310): Store=fdafca090ed7cc34c4ad878aac762e9e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:53,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:53,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:53,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:53:53,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:53,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-04 10:53:53,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fdafca090ed7cc34c4ad878aac762e9e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73266098, jitterRate=0.09174993634223938}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:53:53,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fdafca090ed7cc34c4ad878aac762e9e: 2023-05-04 10:53:53,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e., pid=27, masterSystemTime=1683197633891 2023-05-04 10:53:53,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:53,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:53,921 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=fdafca090ed7cc34c4ad878aac762e9e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:53,921 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683197633921"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197633921"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197633921"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197633921"}]},"ts":"1683197633921"} 2023-05-04 10:53:53,926 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-05-04 10:53:53,926 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528 in 184 msec 2023-05-04 10:53:53,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-05-04 10:53:53,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, ASSIGN in 345 msec 2023-05-04 10:53:53,930 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-04 10:53:53,930 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197633930"}]},"ts":"1683197633930"} 2023-05-04 10:53:53,932 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-05-04 10:53:53,934 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-05-04 10:53:53,936 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 397 msec 2023-05-04 10:53:54,052 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:53:54,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-04 10:53:54,054 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-04 10:53:54,055 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,056 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-04 10:53:54,056 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-04 10:53:54,056 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,056 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,058 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,058 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,058 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,058 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,059 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,059 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:53:54,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,060 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,060 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,060 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:54,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,060 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-04 10:53:54,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:54,060 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-04 10:53:54,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:54,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,061 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-04 10:53:54,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,061 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,061 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-04 10:53:54,062 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-04 10:53:54,061 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-04 10:53:54,062 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-04 10:53:54,062 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-04 10:53:54,062 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-04 10:53:54,062 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-04 10:53:54,062 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,062 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-04 10:53:54,063 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-04 10:53:54,063 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-04 10:53:54,063 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:53:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:53:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-04 10:53:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-04 10:53:54,064 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-04 10:53:54,064 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. started... 2023-05-04 10:53:54,065 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fdafca090ed7cc34c4ad878aac762e9e 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-04 10:53:54,065 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,066 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,066 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,066 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:54,067 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,067 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:54,083 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf1/1739f54b0b3342b28a862c59b7f3d55a 2023-05-04 10:53:54,106 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf2/daab6abcb1764bfe87a743406f93eacf 2023-05-04 10:53:54,113 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf1/1739f54b0b3342b28a862c59b7f3d55a as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/1739f54b0b3342b28a862c59b7f3d55a 2023-05-04 10:53:54,118 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/1739f54b0b3342b28a862c59b7f3d55a, entries=2, sequenceid=6, filesize=4.8 K 2023-05-04 10:53:54,119 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf2/daab6abcb1764bfe87a743406f93eacf as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/daab6abcb1764bfe87a743406f93eacf 2023-05-04 10:53:54,124 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/daab6abcb1764bfe87a743406f93eacf, entries=2, sequenceid=6, filesize=4.8 K 2023-05-04 10:53:54,125 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for fdafca090ed7cc34c4ad878aac762e9e in 60ms, sequenceid=6, compaction requested=false 2023-05-04 10:53:54,125 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-05-04 10:53:54,126 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fdafca090ed7cc34c4ad878aac762e9e: 2023-05-04 10:53:54,126 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:54,126 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:53:54,126 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:53:54,126 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,126 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-04 10:53:54,126 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-04 10:53:54,128 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,128 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:54,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:54,128 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,128 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:54,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:54,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:54,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:54,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-04 10:53:54,131 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4d3abc60[Count = 0] remaining members to acquire global barrier 2023-05-04 10:53:54,131 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-04 10:53:54,131 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,133 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,133 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,133 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,133 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-04 10:53:54,133 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,133 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:53:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,133 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-04 10:53:54,133 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-04 10:53:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,133 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-04 10:53:54,133 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-04 10:53:54,133 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:53:54,133 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-04 10:53:54,133 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:53:54,134 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:53:54,135 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-04 10:53:54,135 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,135 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-04 10:53:54,136 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-04 10:53:54,136 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,136 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-04 10:53:54,136 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-04 10:53:54,136 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,136 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,137 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-04 10:53:54,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:54,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:53:54,141 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-04 10:53:54,141 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-04 10:53:54,141 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:53:54,141 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:53:54,141 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-04 10:53:54,142 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:53:54,143 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,143 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,143 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,143 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:54,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,143 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,144 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,143 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:54,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,144 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,145 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,145 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:54,147 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,147 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,148 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,152 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,152 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,152 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,152 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,152 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,152 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,152 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-04 10:53:54,153 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:53:54,152 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,152 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,153 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,152 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,153 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,154 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-04 10:53:54,154 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,154 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,154 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:53:54,155 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,155 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,155 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,155 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,155 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,255 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:53:54,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:53:54,275 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:53:54,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-04 10:53:54,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-04 10:53:54,279 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-04 10:53:54,279 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,279 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-04 10:53:54,279 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-04 10:53:54,280 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,280 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,281 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,281 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,281 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,282 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,282 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,282 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,283 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:53:54,283 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:54,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,283 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-04 10:53:54,283 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-04 10:53:54,283 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-04 10:53:54,283 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-04 10:53:54,284 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,284 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,284 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-04 10:53:54,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:54,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,284 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-04 10:53:54,284 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:53:54,285 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:53:54,285 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:54,285 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-04 10:53:54,285 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. started... 2023-05-04 10:53:54,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:54,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,285 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-04 10:53:54,285 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-04 10:53:54,285 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-04 10:53:54,285 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fdafca090ed7cc34c4ad878aac762e9e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:53:54,285 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,285 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-04 10:53:54,288 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-04 10:53:54,289 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-04 10:53:54,289 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-04 10:53:54,289 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-04 10:53:54,289 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-04 10:53:54,289 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,289 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-04 10:53:54,289 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-04 10:53:54,290 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-04 10:53:54,291 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,291 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,291 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,292 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:54,292 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,292 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:54,301 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf1/c90c12d375264697a4271e0e631847f7 2023-05-04 10:53:54,315 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf2/2259719df39a4e0597c03ef49f4c911c 2023-05-04 10:53:54,320 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf1/c90c12d375264697a4271e0e631847f7 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/c90c12d375264697a4271e0e631847f7 2023-05-04 10:53:54,325 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/c90c12d375264697a4271e0e631847f7, entries=10, sequenceid=19, filesize=5.0 K 2023-05-04 10:53:54,326 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf2/2259719df39a4e0597c03ef49f4c911c as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/2259719df39a4e0597c03ef49f4c911c 2023-05-04 10:53:54,330 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/2259719df39a4e0597c03ef49f4c911c, entries=10, sequenceid=19, filesize=5.0 K 2023-05-04 10:53:54,332 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fdafca090ed7cc34c4ad878aac762e9e in 47ms, sequenceid=19, compaction requested=false 2023-05-04 10:53:54,332 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fdafca090ed7cc34c4ad878aac762e9e: 2023-05-04 10:53:54,332 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:54,332 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:53:54,332 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:53:54,332 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,332 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-04 10:53:54,332 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-04 10:53:54,334 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,334 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:54,334 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,334 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:54,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:54,336 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-04 10:53:54,336 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7cc74ae3[Count = 0] remaining members to acquire global barrier 2023-05-04 10:53:54,336 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-04 10:53:54,336 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,338 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,338 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,338 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,339 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-04 10:53:54,339 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-04 10:53:54,339 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-04 10:53:54,339 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-04 10:53:54,339 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:53:54,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,339 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:53:54,339 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,339 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:53:54,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,339 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-04 10:53:54,339 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-04 10:53:54,339 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:53:54,340 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-04 10:53:54,340 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,340 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-04 10:53:54,344 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,344 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-04 10:53:54,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:54,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:54,344 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,344 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-04 10:53:54,345 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-04 10:53:54,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:54,346 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,346 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-04 10:53:54,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:54,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:54,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:53:54,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-04 10:53:54,351 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-04 10:53:54,351 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:53:54,351 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:53:54,351 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-04 10:53:54,351 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:53:54,353 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,353 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,353 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:54,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:54,353 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,353 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,353 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,353 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:54,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,353 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,354 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,354 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:54,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,356 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,357 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:54,357 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,360 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,360 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,360 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,360 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,360 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,361 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,361 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,361 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,361 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,361 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,361 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,361 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,361 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,361 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,361 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,361 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,361 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-04 10:53:54,361 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:53:54,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,362 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-04 10:53:54,362 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:53:54,369 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,369 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,448 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-05-04 10:53:54,448 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-05-04 10:53:54,449 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:53:54,449 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-05-04 10:53:54,462 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:53:54,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:53:54,482 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:53:54,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-04 10:53:54,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-04 10:53:54,484 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-04 10:53:54,484 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,485 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-04 10:53:54,485 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-04 10:53:54,486 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,486 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,487 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,487 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,487 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,487 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,488 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,488 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:53:54,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,488 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:54,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,489 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-04 10:53:54,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:54,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:54,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,489 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-04 10:53:54,489 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-04 10:53:54,489 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-04 10:53:54,489 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-04 10:53:54,489 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-04 10:53:54,489 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-04 10:53:54,489 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,489 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-04 10:53:54,490 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-04 10:53:54,490 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-04 10:53:54,489 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-04 10:53:54,490 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-04 10:53:54,490 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-04 10:53:54,490 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-04 10:53:54,491 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-04 10:53:54,491 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,491 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,491 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-04 10:53:54,492 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-04 10:53:54,492 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-04 10:53:54,492 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-04 10:53:54,492 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:53:54,493 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:53:54,493 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:54,493 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. started... 2023-05-04 10:53:54,493 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fdafca090ed7cc34c4ad878aac762e9e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:53:54,494 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,496 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,496 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,496 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:54,496 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,496 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:54,509 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf1/fa3e0c9057cd41358732e33b9a9e98f7 2023-05-04 10:53:54,523 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf2/1210892018ef49139fb716228f4800da 2023-05-04 10:53:54,529 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf1/fa3e0c9057cd41358732e33b9a9e98f7 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/fa3e0c9057cd41358732e33b9a9e98f7 2023-05-04 10:53:54,533 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/fa3e0c9057cd41358732e33b9a9e98f7, entries=10, sequenceid=32, filesize=5.0 K 2023-05-04 10:53:54,534 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf2/1210892018ef49139fb716228f4800da as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/1210892018ef49139fb716228f4800da 2023-05-04 10:53:54,540 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/1210892018ef49139fb716228f4800da, entries=10, sequenceid=32, filesize=5.0 K 2023-05-04 10:53:54,541 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fdafca090ed7cc34c4ad878aac762e9e in 48ms, sequenceid=32, compaction requested=true 2023-05-04 10:53:54,541 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fdafca090ed7cc34c4ad878aac762e9e: 2023-05-04 10:53:54,541 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:54,541 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:53:54,541 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:53:54,541 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,541 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-04 10:53:54,541 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-04 10:53:54,543 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,543 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:54,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:54,543 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,544 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:54,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:54,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:54,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:54,545 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-04 10:53:54,546 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1455e4bf[Count = 0] remaining members to acquire global barrier 2023-05-04 10:53:54,546 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-04 10:53:54,546 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,547 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,547 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,547 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,547 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-04 10:53:54,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,547 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-04 10:53:54,547 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-04 10:53:54,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,547 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-04 10:53:54,547 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-04 10:53:54,547 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-04 10:53:54,547 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:53:54,547 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,547 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:53:54,547 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:53:54,547 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:53:54,550 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,550 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-04 10:53:54,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:54,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:54,550 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,550 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-04 10:53:54,551 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-04 10:53:54,551 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-04 10:53:54,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:54,551 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,551 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,551 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-04 10:53:54,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:54,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,551 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-04 10:53:54,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:54,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:53:54,554 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-04 10:53:54,554 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-04 10:53:54,554 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:53:54,554 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:53:54,554 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-04 10:53:54,554 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:53:54,556 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,556 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,556 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:54,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:54,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,556 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,556 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,556 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,557 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,557 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:54,557 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,557 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,559 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,559 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:54,560 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:54,563 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,563 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,563 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,563 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,563 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,563 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,563 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,563 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,563 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-04 10:53:54,563 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:53:54,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,564 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitWithoutPONR because node does not exist (not an error) 2023-05-04 10:53:54,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,563 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,564 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,564 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-04 10:53:54,564 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:53:54,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,564 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,564 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,565 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,565 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,565 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,565 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,565 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,565 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,664 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:53:54,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:53:54,686 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:53:54,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-04 10:53:54,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-04 10:53:54,688 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-04 10:53:54,688 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,689 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-04 10:53:54,689 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-04 10:53:54,689 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,689 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,692 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,692 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,692 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,692 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,692 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,692 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:53:54,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:54,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:54,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,693 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-04 10:53:54,693 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-04 10:53:54,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:54,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,693 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-04 10:53:54,694 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-04 10:53:54,694 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-04 10:53:54,694 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-04 10:53:54,694 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-04 10:53:54,694 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-04 10:53:54,694 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-04 10:53:54,694 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,694 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-04 10:53:54,695 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-04 10:53:54,695 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-04 10:53:54,694 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,695 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-04 10:53:54,696 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-04 10:53:54,696 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-04 10:53:54,695 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-04 10:53:54,696 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-04 10:53:54,696 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,696 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-04 10:53:54,697 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-04 10:53:54,699 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:53:54,699 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:53:54,699 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:54,699 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. started... 2023-05-04 10:53:54,700 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fdafca090ed7cc34c4ad878aac762e9e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:53:54,705 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,706 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,706 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,706 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:54,706 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,706 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:54,719 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf1/575b5478fea041afa8e27fdfaa03ac95 2023-05-04 10:53:54,734 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf2/6a9725bd73054bfeb1d17dbed27f1672 2023-05-04 10:53:54,740 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf1/575b5478fea041afa8e27fdfaa03ac95 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/575b5478fea041afa8e27fdfaa03ac95 2023-05-04 10:53:54,746 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/575b5478fea041afa8e27fdfaa03ac95, entries=10, sequenceid=45, filesize=5.0 K 2023-05-04 10:53:54,747 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf2/6a9725bd73054bfeb1d17dbed27f1672 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/6a9725bd73054bfeb1d17dbed27f1672 2023-05-04 10:53:54,752 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/6a9725bd73054bfeb1d17dbed27f1672, entries=10, sequenceid=45, filesize=5.0 K 2023-05-04 10:53:54,753 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fdafca090ed7cc34c4ad878aac762e9e in 53ms, sequenceid=45, compaction requested=true 2023-05-04 10:53:54,753 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fdafca090ed7cc34c4ad878aac762e9e: 2023-05-04 10:53:54,754 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:54,754 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:53:54,754 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:53:54,754 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,754 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-04 10:53:54,754 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-04 10:53:54,757 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,757 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:54,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:54,757 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,757 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:54,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:54,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:54,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:54,759 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-04 10:53:54,759 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19786f6b[Count = 0] remaining members to acquire global barrier 2023-05-04 10:53:54,759 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-04 10:53:54,759 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,760 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,761 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,760 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,761 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-04 10:53:54,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,761 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,761 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:53:54,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,761 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-04 10:53:54,761 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-04 10:53:54,761 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-04 10:53:54,761 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:53:54,761 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-04 10:53:54,761 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:53:54,761 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-04 10:53:54,761 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:53:54,763 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-04 10:53:54,763 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,763 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-04 10:53:54,763 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-04 10:53:54,763 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-04 10:53:54,763 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,763 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,763 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,763 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-04 10:53:54,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:54,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:54,763 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-04 10:53:54,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:54,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:54,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:54,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:53:54,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-04 10:53:54,768 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-04 10:53:54,768 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:53:54,768 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:53:54,768 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-04 10:53:54,768 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:53:54,770 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,770 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,770 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,770 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,770 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,770 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:54,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:54,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,770 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,771 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:54,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,772 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,773 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:54,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,775 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:54,775 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,775 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,779 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,779 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,779 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,779 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,779 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,779 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,779 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,779 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,779 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,779 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,779 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-04 10:53:54,780 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:53:54,779 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,779 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,779 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,780 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,780 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,780 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-04 10:53:54,780 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,780 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,780 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,780 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:53:54,881 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:53:54,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:53:54,903 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:53:54,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-04 10:53:54,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-04 10:53:54,906 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-04 10:53:54,906 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,906 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-04 10:53:54,906 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-04 10:53:54,906 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,906 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,908 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,908 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,908 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,908 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,908 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,908 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:53:54,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:54,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:54,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,910 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-04 10:53:54,909 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-04 10:53:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:54,910 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-04 10:53:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,910 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-04 10:53:54,910 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-04 10:53:54,910 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-04 10:53:54,910 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,910 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-04 10:53:54,910 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-04 10:53:54,910 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-04 10:53:54,910 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-04 10:53:54,910 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-04 10:53:54,910 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-04 10:53:54,910 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,910 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-04 10:53:54,910 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-04 10:53:54,911 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-04 10:53:54,912 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,912 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,912 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:54,914 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,914 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,914 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:54,916 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-04 10:53:54,916 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-04 10:53:54,916 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-04 10:53:54,916 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:54,916 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-04 10:53:54,916 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:53:54,916 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:53:54,916 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:54,916 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. started... 2023-05-04 10:53:54,917 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fdafca090ed7cc34c4ad878aac762e9e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:53:54,932 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf1/2eede810dbbb42ffbb5ade091a470416 2023-05-04 10:53:54,947 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf2/cffd4e95da4849b8bfb11d0b9ae14fae 2023-05-04 10:53:54,952 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf1/2eede810dbbb42ffbb5ade091a470416 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/2eede810dbbb42ffbb5ade091a470416 2023-05-04 10:53:54,957 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/2eede810dbbb42ffbb5ade091a470416, entries=10, sequenceid=58, filesize=5.0 K 2023-05-04 10:53:54,958 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf2/cffd4e95da4849b8bfb11d0b9ae14fae as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/cffd4e95da4849b8bfb11d0b9ae14fae 2023-05-04 10:53:54,962 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/cffd4e95da4849b8bfb11d0b9ae14fae, entries=10, sequenceid=58, filesize=5.0 K 2023-05-04 10:53:54,964 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fdafca090ed7cc34c4ad878aac762e9e in 48ms, sequenceid=58, compaction requested=true 2023-05-04 10:53:54,964 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fdafca090ed7cc34c4ad878aac762e9e: 2023-05-04 10:53:54,964 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:54,964 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:53:54,964 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:53:54,964 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,964 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-04 10:53:54,964 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-04 10:53:54,966 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,966 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:54,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:54,966 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,966 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:54,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:54,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:54,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:54,968 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-04 10:53:54,968 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1391998[Count = 0] remaining members to acquire global barrier 2023-05-04 10:53:54,969 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-04 10:53:54,969 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,970 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-04 10:53:54,970 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,970 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:53:54,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,970 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-04 10:53:54,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,970 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:53:54,970 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-04 10:53:54,970 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-04 10:53:54,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:54,970 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:53:54,970 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-04 10:53:54,970 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-04 10:53:54,970 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:53:54,972 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-04 10:53:54,972 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,972 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-04 10:53:54,973 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-04 10:53:54,973 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,973 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-04 10:53:54,974 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,974 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-04 10:53:54,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:54,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:54,974 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,974 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-04 10:53:54,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:54,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:54,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:54,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:53:54,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-04 10:53:54,977 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-04 10:53:54,977 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:53:54,977 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:53:54,977 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-04 10:53:54,977 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:53:54,979 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,979 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,979 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,979 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,979 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:54,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:54,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,979 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,979 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,979 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:54,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,980 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:54,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,980 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:54,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:54,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,982 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:54,982 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:54,982 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:54,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:54,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,999 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:54,999 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,999 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:54,999 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,999 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:54,999 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:54,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:54,999 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,999 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-04 10:53:54,999 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:53:54,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:54,999 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:54,999 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:55,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:55,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:54,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:55,000 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-04 10:53:54,999 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:55,000 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:53:55,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:55,000 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:55,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:55,000 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:55,000 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,000 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,000 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:55,001 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:55,001 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,001 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:55,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:55,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:55,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:55,100 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:53:55,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:53:55,124 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:53:55,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-04 10:53:55,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-04 10:53:55,127 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-04 10:53:55,127 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:55,127 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-04 10:53:55,127 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-04 10:53:55,127 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,127 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:55,129 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:55,129 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:55,129 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:55,129 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:55,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:55,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:55,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:55,129 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,129 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:53:55,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:55,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:55,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:55,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:55,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:55,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:55,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:55,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:55,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:55,130 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-04 10:53:55,130 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-04 10:53:55,131 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-04 10:53:55,131 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-04 10:53:55,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:53:55,131 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-04 10:53:55,131 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:55,131 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-04 10:53:55,131 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-04 10:53:55,131 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-04 10:53:55,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:55,131 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-04 10:53:55,131 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:53:55,131 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:53:55,131 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-04 10:53:55,131 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:55,131 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:55,132 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-04 10:53:55,132 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. started... 2023-05-04 10:53:55,132 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-04 10:53:55,132 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-04 10:53:55,132 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fdafca090ed7cc34c4ad878aac762e9e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:53:55,132 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-04 10:53:55,132 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-04 10:53:55,132 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-04 10:53:55,132 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:53:55,132 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-04 10:53:55,132 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-04 10:53:55,132 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-04 10:53:55,134 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,134 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,134 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,134 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:55,134 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,134 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:55,147 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf1/6fee68f6be104910b531998452be1707 2023-05-04 10:53:55,568 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf2/599c55505d03422d8dfd8e2522e32fa9 2023-05-04 10:53:55,574 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf1/6fee68f6be104910b531998452be1707 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/6fee68f6be104910b531998452be1707 2023-05-04 10:53:55,580 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/6fee68f6be104910b531998452be1707, entries=10, sequenceid=71, filesize=5.0 K 2023-05-04 10:53:55,581 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf2/599c55505d03422d8dfd8e2522e32fa9 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/599c55505d03422d8dfd8e2522e32fa9 2023-05-04 10:53:55,587 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/599c55505d03422d8dfd8e2522e32fa9, entries=10, sequenceid=71, filesize=5.0 K 2023-05-04 10:53:55,588 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fdafca090ed7cc34c4ad878aac762e9e in 456ms, sequenceid=71, compaction requested=true 2023-05-04 10:53:55,588 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fdafca090ed7cc34c4ad878aac762e9e: 2023-05-04 10:53:55,588 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:55,588 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:53:55,588 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:53:55,588 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,588 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-04 10:53:55,588 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-04 10:53:55,592 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,592 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:55,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:55,592 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,592 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:53:55,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:55,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:55,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:55,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:55,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:55,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:55,594 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-04 10:53:55,594 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-04 10:53:55,594 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@476fdb0e[Count = 0] remaining members to acquire global barrier 2023-05-04 10:53:55,594 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,595 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,595 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,595 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,596 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,596 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:53:55,596 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-04 10:53:55,596 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-04 10:53:55,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,596 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:53:55,596 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-04 10:53:55,596 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-04 10:53:55,596 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:53:55,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,596 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-04 10:53:55,596 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-04 10:53:55,596 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:53:55,597 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-04 10:53:55,597 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:55,597 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-04 10:53:55,598 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,598 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-04 10:53:55,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:55,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:55,598 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-04 10:53:55,598 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:55,598 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:55,598 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-04 10:53:55,598 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-04 10:53:55,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:55,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:55,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:55,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:55,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:55,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:55,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:55,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:55,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:55,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:53:55,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-04 10:53:55,601 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-04 10:53:55,601 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:53:55,601 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:53:55,601 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-04 10:53:55,601 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:53:55,603 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,603 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,603 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,603 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,604 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:55,604 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:55,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:53:55,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:53:55,603 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:55,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:55,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:55,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:53:55,604 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:55,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:55,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:55,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:55,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:55,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:55,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,605 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:55,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:53:55,605 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:55,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:55,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:55,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:53:55,607 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:55,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-04 10:53:55,607 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:55,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:55,607 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:55,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,610 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,610 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:55,610 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:55,610 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:55,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:55,610 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-04 10:53:55,610 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:55,610 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:53:55,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:55,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:55,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:55,610 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:55,610 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:53:55,610 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:53:55,610 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:55,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:53:55,610 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:55,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:53:55,610 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-04 10:53:55,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:55,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:55,610 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-04 10:53:55,611 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:53:55,610 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:53:55,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:55,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:55,610 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:53:55,611 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:53:55,611 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:53:55,611 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:53:55,611 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-04 10:53:55,611 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-04 10:53:55,711 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:53:55,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:53:55,975 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-05-04 10:53:55,976 DEBUG [Listener at localhost/35483] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:55,978 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:55,979 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:55,979 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:55,979 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:55,980 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:53:55,980 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:55,980 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:53:55,980 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:53:56,228 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,229 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:53:56,230 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:53:56,230 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:53:56,232 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-05-04 10:53:56,232 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:53:56,232 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:53:56,233 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:53:56,233 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:53:56,234 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:53:56,237 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-05-04 10:53:56,238 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:53:56,239 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:53:56,239 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:53:56,240 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:53:56,240 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:56,241 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:56,241 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:56,242 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:56,242 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:53:56,242 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,243 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:53:56,243 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,243 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,243 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,243 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-05-04 10:53:56,244 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:53:56,245 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:53:56,246 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197666246 2023-05-04 10:53:56,247 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,247 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:53:56,252 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:53:56,253 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=fdafca090ed7cc34c4ad878aac762e9e, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42285,1683197625528, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528, openSeqNum=2 2023-05-04 10:53:56,253 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:53:56,253 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197696253 2023-05-04 10:53:56,253 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197756253 2023-05-04 10:53:56,253 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-05-04 10:53:56,253 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:53:56,255 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:53:56,256 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:53:56,256 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:53:56,503 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,503 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:53:56,503 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:53:56,504 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:53:56,504 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:53:56,504 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:53:56,504 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:53:56,504 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:53:56,504 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:53:56,505 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:53:56,507 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-05-04 10:53:56,508 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:53:56,509 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:53:56,509 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:53:56,509 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:53:56,509 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:56,510 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:56,510 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:56,510 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:56,510 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:53:56,511 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,511 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:53:56,511 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,511 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,511 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,511 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-04 10:53:56,512 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:53:56,512 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:53:56,516 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197666516 2023-05-04 10:53:56,517 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:53:56,517 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,520 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:53:56,521 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=fdafca090ed7cc34c4ad878aac762e9e, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42285,1683197625528, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528, openSeqNum=2 2023-05-04 10:53:56,521 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:53:56,521 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197696521 2023-05-04 10:53:56,521 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197756521 2023-05-04 10:53:56,521 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-04 10:53:56,521 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:53:56,523 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN}] 2023-05-04 10:53:56,523 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:53:56,524 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:53:56,524 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:53:56,771 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,771 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:53:56,771 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:53:56,772 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:53:56,772 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:53:56,772 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:53:56,772 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:53:56,772 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:53:56,772 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:53:56,773 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:53:56,774 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-05-04 10:53:56,775 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:53:56,775 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:53:56,775 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:53:56,775 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:53:56,776 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:56,776 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:56,777 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:56,777 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:56,777 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:53:56,777 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,778 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:53:56,778 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,778 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,778 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,778 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-04 10:53:56,779 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:53:56,779 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:53:56,779 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197666779 2023-05-04 10:53:56,781 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:53:56,781 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:56,784 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:53:56,785 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=fdafca090ed7cc34c4ad878aac762e9e, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,42285,1683197625528, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528, openSeqNum=2 2023-05-04 10:53:56,785 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:53:56,785 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197696785 2023-05-04 10:53:56,785 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197756785 2023-05-04 10:53:56,785 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:53:56,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN}] 2023-05-04 10:53:56,785 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:53:56,788 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN 2023-05-04 10:53:56,789 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=fdafca090ed7cc34c4ad878aac762e9e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:56,789 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683197636789"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197636789"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197636789"}]},"ts":"1683197636789"} 2023-05-04 10:53:56,791 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:53:56,791 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:53:56,791 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN 2023-05-04 10:53:56,791 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:53:56,791 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:53:57,035 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,035 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:53:57,035 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:53:57,036 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:53:57,037 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:53:57,037 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:53:57,037 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:53:57,037 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:53:57,037 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:53:57,038 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:53:57,040 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-04 10:53:57,040 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-04 10:53:57,040 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:53:57,041 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:53:57,041 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:53:57,041 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:53:57,041 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:57,042 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:57,042 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:57,042 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:57,042 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:53:57,043 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,044 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN 2023-05-04 10:53:57,044 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:53:57,044 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,044 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,044 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,044 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:53:57,044 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN 2023-05-04 10:53:57,044 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-04 10:53:57,045 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:53:57,045 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:53:57,045 INFO [Listener at localhost/35483] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e to restore RIT 2023-05-04 10:53:57,046 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197667046 2023-05-04 10:53:57,046 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:53:57,047 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN 2023-05-04 10:53:57,049 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:53:57,049 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=fdafca090ed7cc34c4ad878aac762e9e, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,42285,1683197625528, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528, openSeqNum=2 2023-05-04 10:53:57,049 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:53:57,049 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197697049 2023-05-04 10:53:57,049 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197757049 2023-05-04 10:53:57,050 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:53:57,050 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=fdafca090ed7cc34c4ad878aac762e9e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:57,050 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683197637050"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197637050"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197637050"}]},"ts":"1683197637050"} 2023-05-04 10:53:57,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:53:57,052 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:53:57,206 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:57,206 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fdafca090ed7cc34c4ad878aac762e9e, disabling compactions & flushes 2023-05-04 10:53:57,206 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:57,206 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:57,206 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. after waiting 0 ms 2023-05-04 10:53:57,206 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:57,206 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing fdafca090ed7cc34c4ad878aac762e9e 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-04 10:53:57,218 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf1/bcc61811539a4e10a1ee403686d11536 2023-05-04 10:53:57,237 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf2/8747423eb5444c3cac8b7f007a8c4ba8 2023-05-04 10:53:57,243 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf1/bcc61811539a4e10a1ee403686d11536 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/bcc61811539a4e10a1ee403686d11536 2023-05-04 10:53:57,248 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/bcc61811539a4e10a1ee403686d11536, entries=8, sequenceid=82, filesize=5.0 K 2023-05-04 10:53:57,249 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/.tmp/cf2/8747423eb5444c3cac8b7f007a8c4ba8 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/8747423eb5444c3cac8b7f007a8c4ba8 2023-05-04 10:53:57,254 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/8747423eb5444c3cac8b7f007a8c4ba8, entries=8, sequenceid=82, filesize=5.0 K 2023-05-04 10:53:57,255 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for fdafca090ed7cc34c4ad878aac762e9e in 49ms, sequenceid=82, compaction requested=true 2023-05-04 10:53:57,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-04 10:53:57,279 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:53:57,279 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. 2023-05-04 10:53:57,279 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fdafca090ed7cc34c4ad878aac762e9e: 2023-05-04 10:53:57,282 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:57,282 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=fdafca090ed7cc34c4ad878aac762e9e, regionState=CLOSED 2023-05-04 10:53:57,283 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683197637282"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197637282"}]},"ts":"1683197637282"} 2023-05-04 10:53:57,284 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:53:57,284 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:57,284 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:53:57,284 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:53:57,300 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,300 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:53:57,300 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:53:57,300 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:53:57,301 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:53:57,301 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:53:57,301 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:53:57,301 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:53:57,301 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:53:57,302 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:53:57,304 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-04 10:53:57,304 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-05-04 10:53:57,304 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-05-04 10:53:57,304 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:53:57,305 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:53:57,305 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:53:57,305 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:53:57,306 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:57,306 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:57,307 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:57,307 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:57,307 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:53:57,308 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,309 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN 2023-05-04 10:53:57,309 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:57,309 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:53:57,309 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,309 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,309 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,309 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:53:57,309 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN 2023-05-04 10:53:57,310 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:53:57,310 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-04 10:53:57,310 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:53:57,311 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:53:57,311 INFO [Listener at localhost/35483] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e to restore RIT 2023-05-04 10:53:57,311 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197667311 2023-05-04 10:53:57,312 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:57,312 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:53:57,315 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:53:57,316 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=fdafca090ed7cc34c4ad878aac762e9e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42285,1683197625528, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528, openSeqNum=2 2023-05-04 10:53:57,316 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:53:57,316 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197697316 2023-05-04 10:53:57,316 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197757316 2023-05-04 10:53:57,316 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:53:57,316 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=fdafca090ed7cc34c4ad878aac762e9e, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:57,316 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683197637316"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197637316"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197637316"}]},"ts":"1683197637316"} 2023-05-04 10:53:57,318 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:53:57,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-05-04 10:53:57,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528 in 266 msec 2023-05-04 10:53:57,321 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:53:57,321 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN 2023-05-04 10:53:57,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:53:57,321 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:53:57,566 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,566 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:53:57,566 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:53:57,567 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:53:57,567 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:53:57,567 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:53:57,567 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:53:57,567 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:53:57,567 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:53:57,568 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:53:57,570 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-04 10:53:57,570 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-05-04 10:53:57,570 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:53:57,570 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:53:57,571 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:53:57,571 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:53:57,571 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:53:57,572 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:57,572 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:57,573 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:57,573 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:57,574 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:53:57,574 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,575 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN 2023-05-04 10:53:57,575 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:57,575 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:53:57,575 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,575 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,575 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,575 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:53:57,576 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:53:57,576 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-04 10:53:57,576 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:53:57,577 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:53:57,577 INFO [Listener at localhost/35483] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e to restore RIT 2023-05-04 10:53:57,578 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197667578 2023-05-04 10:53:57,579 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN 2023-05-04 10:53:57,579 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:53:57,581 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:53:57,581 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=fdafca090ed7cc34c4ad878aac762e9e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42285,1683197625528, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528, openSeqNum=2 2023-05-04 10:53:57,581 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:53:57,581 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197697581 2023-05-04 10:53:57,581 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197757581 2023-05-04 10:53:57,582 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-04 10:53:57,582 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN 2023-05-04 10:53:57,583 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:53:57,586 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-05-04 10:53:57,586 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN in 798 msec 2023-05-04 10:53:57,589 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:53:57,589 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,589 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:53:57,589 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:53:57,718 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-04 10:53:57,832 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,832 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:53:57,832 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:53:57,832 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:53:57,833 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:53:57,833 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:53:57,833 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:53:57,833 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:53:57,833 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:53:57,834 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:53:57,835 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-04 10:53:57,835 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:53:57,835 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:53:57,836 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:53:57,836 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:53:57,836 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:53:57,836 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:53:57,837 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:57,837 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:57,837 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:57,837 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:57,838 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:53:57,838 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,839 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN 2023-05-04 10:53:57,839 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:57,839 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:53:57,839 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:53:57,839 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:53:57,839 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-04 10:53:57,840 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:53:57,840 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:53:57,840 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197667840 2023-05-04 10:53:57,842 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,842 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:53:57,844 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:53:57,845 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=fdafca090ed7cc34c4ad878aac762e9e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42285,1683197625528, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528, openSeqNum=2 2023-05-04 10:53:57,845 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:53:57,845 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197697845 2023-05-04 10:53:57,845 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197757845 2023-05-04 10:53:57,845 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:53:57,845 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,845 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,845 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:57,850 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:53:57,860 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=fdafca090ed7cc34c4ad878aac762e9e, threads=14 2023-05-04 10:53:57,861 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/1210892018ef49139fb716228f4800da for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:57,861 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/2259719df39a4e0597c03ef49f4c911c for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:57,861 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/599c55505d03422d8dfd8e2522e32fa9 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:57,861 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/6a9725bd73054bfeb1d17dbed27f1672 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:57,861 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/8747423eb5444c3cac8b7f007a8c4ba8 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:57,861 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/cffd4e95da4849b8bfb11d0b9ae14fae for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:57,861 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/daab6abcb1764bfe87a743406f93eacf for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:57,861 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/1739f54b0b3342b28a862c59b7f3d55a for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:57,861 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/2eede810dbbb42ffbb5ade091a470416 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:57,862 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/575b5478fea041afa8e27fdfaa03ac95 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:57,862 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/6fee68f6be104910b531998452be1707 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:57,862 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/bcc61811539a4e10a1ee403686d11536 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:57,862 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/c90c12d375264697a4271e0e631847f7 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:57,863 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/fa3e0c9057cd41358732e33b9a9e98f7 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,046 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/1210892018ef49139fb716228f4800da for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,059 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/8747423eb5444c3cac8b7f007a8c4ba8 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,060 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/575b5478fea041afa8e27fdfaa03ac95 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,065 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/1739f54b0b3342b28a862c59b7f3d55a for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,075 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/bcc61811539a4e10a1ee403686d11536 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,080 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/c90c12d375264697a4271e0e631847f7 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,092 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/cffd4e95da4849b8bfb11d0b9ae14fae for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,100 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/2259719df39a4e0597c03ef49f4c911c for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,112 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/fa3e0c9057cd41358732e33b9a9e98f7 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,112 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/6a9725bd73054bfeb1d17dbed27f1672 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,114 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/6fee68f6be104910b531998452be1707 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,114 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/599c55505d03422d8dfd8e2522e32fa9 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,115 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/daab6abcb1764bfe87a743406f93eacf for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,117 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/2eede810dbbb42ffbb5ade091a470416 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,117 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region fdafca090ed7cc34c4ad878aac762e9e Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-04 10:53:58,162 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:53:58,162 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:58,162 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:53:58,162 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:53:58,345 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:58,345 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:53:58,345 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:53:58,345 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:53:58,346 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:53:58,347 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:53:58,347 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:53:58,347 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:53:58,347 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:53:58,348 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:53:58,349 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-05-04 10:53:58,349 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:53:58,349 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:53:58,349 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:53:58,350 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:53:58,350 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:53:58,350 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:53:58,351 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:58,351 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:58,352 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:58,352 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:58,352 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:53:58,353 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:58,353 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN 2023-05-04 10:53:58,353 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:58,353 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:53:58,353 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:58,353 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:58,353 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:58,354 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:53:58,354 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:53:58,354 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-04 10:53:58,354 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:53:58,355 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:53:58,355 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197668355 2023-05-04 10:53:58,357 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:53:58,357 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:58,359 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:53:58,360 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=fdafca090ed7cc34c4ad878aac762e9e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42285,1683197625528, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528, openSeqNum=2 2023-05-04 10:53:58,360 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:53:58,360 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197698360 2023-05-04 10:53:58,360 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197758360 2023-05-04 10:53:58,360 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:53:58,370 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=fdafca090ed7cc34c4ad878aac762e9e, threads=14 2023-05-04 10:53:58,372 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/1210892018ef49139fb716228f4800da for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,372 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/2259719df39a4e0597c03ef49f4c911c for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,372 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/599c55505d03422d8dfd8e2522e32fa9 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,373 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/6a9725bd73054bfeb1d17dbed27f1672 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,373 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/8747423eb5444c3cac8b7f007a8c4ba8 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,373 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/cffd4e95da4849b8bfb11d0b9ae14fae for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,373 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/daab6abcb1764bfe87a743406f93eacf for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,376 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/1739f54b0b3342b28a862c59b7f3d55a for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,376 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/2eede810dbbb42ffbb5ade091a470416 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,380 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/575b5478fea041afa8e27fdfaa03ac95 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,380 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/6fee68f6be104910b531998452be1707 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,380 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/bcc61811539a4e10a1ee403686d11536 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,380 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/c90c12d375264697a4271e0e631847f7 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,380 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/fa3e0c9057cd41358732e33b9a9e98f7 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,574 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/2eede810dbbb42ffbb5ade091a470416 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,597 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/599c55505d03422d8dfd8e2522e32fa9 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,628 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/1739f54b0b3342b28a862c59b7f3d55a for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,650 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/6fee68f6be104910b531998452be1707 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,650 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/1210892018ef49139fb716228f4800da for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,655 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/8747423eb5444c3cac8b7f007a8c4ba8 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,656 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/c90c12d375264697a4271e0e631847f7 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,657 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/2259719df39a4e0597c03ef49f4c911c for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,667 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/575b5478fea041afa8e27fdfaa03ac95 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,669 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/6a9725bd73054bfeb1d17dbed27f1672 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,671 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/bcc61811539a4e10a1ee403686d11536 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,672 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/cffd4e95da4849b8bfb11d0b9ae14fae for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:58,999 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/daab6abcb1764bfe87a743406f93eacf for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:59,018 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/fa3e0c9057cd41358732e33b9a9e98f7 for region: fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:53:59,019 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region fdafca090ed7cc34c4ad878aac762e9e Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-04 10:53:59,059 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:53:59,065 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-04 10:53:59,068 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-04 10:53:59,068 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:53:59,068 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:53:59,068 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:53:59,110 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,111 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:53:59,111 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:53:59,111 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:53:59,111 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:53:59,111 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:53:59,111 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:53:59,111 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:53:59,111 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:53:59,112 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:53:59,113 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-05-04 10:53:59,113 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:53:59,114 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:53:59,114 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:53:59,115 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:53:59,115 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:53:59,115 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:53:59,115 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:59,116 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:59,116 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:59,116 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:59,116 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:53:59,117 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,117 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN 2023-05-04 10:53:59,117 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:59,117 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:53:59,117 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,118 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,118 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,118 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:53:59,118 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:53:59,118 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-04 10:53:59,119 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:53:59,119 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:53:59,119 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197669119 2023-05-04 10:53:59,121 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:53:59,121 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,124 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:53:59,124 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=fdafca090ed7cc34c4ad878aac762e9e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42285,1683197625528, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528, openSeqNum=2 2023-05-04 10:53:59,124 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:53:59,124 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197699124 2023-05-04 10:53:59,124 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197759124 2023-05-04 10:53:59,124 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:53:59,127 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:53:59,129 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:53:59,129 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:53:59,129 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:53:59,374 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:53:59,375 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:53:59,375 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:53:59,375 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:53:59,375 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:53:59,375 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:53:59,375 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:53:59,375 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:53:59,375 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:53:59,376 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:53:59,377 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-05-04 10:53:59,377 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:53:59,377 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:53:59,377 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:53:59,378 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:53:59,378 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:53:59,378 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:53:59,378 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:59,379 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:53:59,379 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:59,379 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:53:59,379 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:53:59,380 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,380 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN 2023-05-04 10:53:59,380 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:59,380 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:53:59,380 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,380 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,380 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,380 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=fdafca090ed7cc34c4ad878aac762e9e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:53:59,380 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure fdafca090ed7cc34c4ad878aac762e9e, server=jenkins-hbase4.apache.org,42285,1683197625528 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:53:59,381 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-04 10:53:59,381 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:53:59,382 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:53:59,382 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197669382 2023-05-04 10:53:59,383 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,383 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:53:59,385 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:53:59,386 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=fdafca090ed7cc34c4ad878aac762e9e, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,42285,1683197625528, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528, openSeqNum=2 2023-05-04 10:53:59,386 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:53:59,386 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197699386 2023-05-04 10:53:59,386 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197759386 2023-05-04 10:53:59,386 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-04 10:53:59,389 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683197639389"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1683197639389"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1683197639389"}]},"ts":"1683197639389"} 2023-05-04 10:53:59,389 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197639389"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197639389"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197639389"}]},"ts":"1683197639389"} 2023-05-04 10:53:59,390 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197639389"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197639389"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197639389"}]},"ts":"1683197639389"} 2023-05-04 10:53:59,395 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4110ba1eeda845d33e575cf303f91707, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=9db21ffe8cb46d9683ca276a900101ec, ASSIGN}] 2023-05-04 10:53:59,396 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=9db21ffe8cb46d9683ca276a900101ec, ASSIGN 2023-05-04 10:53:59,398 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=9db21ffe8cb46d9683ca276a900101ec, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42285,1683197625528; forceNewPlan=false, retain=false 2023-05-04 10:53:59,400 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4110ba1eeda845d33e575cf303f91707, ASSIGN 2023-05-04 10:53:59,400 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4110ba1eeda845d33e575cf303f91707, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42285,1683197625528; forceNewPlan=false, retain=false 2023-05-04 10:53:59,549 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-04 10:53:59,550 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=4110ba1eeda845d33e575cf303f91707, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:59,551 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197639550"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197639550"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197639550"}]},"ts":"1683197639550"} 2023-05-04 10:53:59,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure 4110ba1eeda845d33e575cf303f91707, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:53:59,556 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=9db21ffe8cb46d9683ca276a900101ec, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:59,556 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197639555"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197639555"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197639555"}]},"ts":"1683197639555"} 2023-05-04 10:53:59,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure 9db21ffe8cb46d9683ca276a900101ec, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:53:59,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec. 2023-05-04 10:53:59,709 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9db21ffe8cb46d9683ca276a900101ec, NAME => 'testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.', STARTKEY => '41', ENDKEY => ''} 2023-05-04 10:53:59,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:53:59,709 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,709 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:53:59,709 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,709 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,711 INFO [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,712 DEBUG [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1 2023-05-04 10:53:59,712 DEBUG [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1 2023-05-04 10:53:59,712 INFO [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9db21ffe8cb46d9683ca276a900101ec columnFamilyName cf1 2023-05-04 10:53:59,729 DEBUG [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/1739f54b0b3342b28a862c59b7f3d55a.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/1739f54b0b3342b28a862c59b7f3d55a-top 2023-05-04 10:53:59,734 DEBUG [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/2eede810dbbb42ffbb5ade091a470416.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/2eede810dbbb42ffbb5ade091a470416-top 2023-05-04 10:53:59,738 DEBUG [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/575b5478fea041afa8e27fdfaa03ac95.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/575b5478fea041afa8e27fdfaa03ac95-top 2023-05-04 10:53:59,743 DEBUG [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/6fee68f6be104910b531998452be1707.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/6fee68f6be104910b531998452be1707-top 2023-05-04 10:53:59,748 DEBUG [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/bcc61811539a4e10a1ee403686d11536.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/bcc61811539a4e10a1ee403686d11536-top 2023-05-04 10:53:59,754 DEBUG [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/c90c12d375264697a4271e0e631847f7.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/c90c12d375264697a4271e0e631847f7-top 2023-05-04 10:53:59,759 DEBUG [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/fa3e0c9057cd41358732e33b9a9e98f7.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/fa3e0c9057cd41358732e33b9a9e98f7-top 2023-05-04 10:53:59,759 INFO [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] regionserver.HStore(310): Store=9db21ffe8cb46d9683ca276a900101ec/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:59,759 INFO [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,760 DEBUG [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2 2023-05-04 10:53:59,760 DEBUG [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2 2023-05-04 10:53:59,761 INFO [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9db21ffe8cb46d9683ca276a900101ec columnFamilyName cf2 2023-05-04 10:53:59,777 DEBUG [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/1210892018ef49139fb716228f4800da.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/1210892018ef49139fb716228f4800da-top 2023-05-04 10:53:59,781 DEBUG [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/2259719df39a4e0597c03ef49f4c911c.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/2259719df39a4e0597c03ef49f4c911c-top 2023-05-04 10:53:59,786 DEBUG [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/599c55505d03422d8dfd8e2522e32fa9.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/599c55505d03422d8dfd8e2522e32fa9-top 2023-05-04 10:53:59,791 DEBUG [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/6a9725bd73054bfeb1d17dbed27f1672.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/6a9725bd73054bfeb1d17dbed27f1672-top 2023-05-04 10:53:59,796 DEBUG [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/8747423eb5444c3cac8b7f007a8c4ba8.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/8747423eb5444c3cac8b7f007a8c4ba8-top 2023-05-04 10:53:59,800 DEBUG [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/cffd4e95da4849b8bfb11d0b9ae14fae.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/cffd4e95da4849b8bfb11d0b9ae14fae-top 2023-05-04 10:53:59,805 DEBUG [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/daab6abcb1764bfe87a743406f93eacf.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/daab6abcb1764bfe87a743406f93eacf-top 2023-05-04 10:53:59,805 INFO [StoreOpener-9db21ffe8cb46d9683ca276a900101ec-1] regionserver.HStore(310): Store=9db21ffe8cb46d9683ca276a900101ec/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:59,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,809 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:53:59,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:53:59,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 9db21ffe8cb46d9683ca276a900101ec; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73970379, jitterRate=0.1022445410490036}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:53:59,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 9db21ffe8cb46d9683ca276a900101ec: 2023-05-04 10:53:59,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec., pid=34, masterSystemTime=1683197639704 2023-05-04 10:53:59,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:53:59,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-04 10:53:59,813 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:53:59,813 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:53:59,813 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec. 2023-05-04 10:53:59,813 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1912): 9db21ffe8cb46d9683ca276a900101ec/cf2 is initiating minor compaction (all files) 2023-05-04 10:53:59,813 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec. 2023-05-04 10:53:59,813 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1912): 9db21ffe8cb46d9683ca276a900101ec/cf1 is initiating minor compaction (all files) 2023-05-04 10:53:59,813 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 9db21ffe8cb46d9683ca276a900101ec/cf2 in testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec. 2023-05-04 10:53:59,813 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 9db21ffe8cb46d9683ca276a900101ec/cf1 in testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec. 2023-05-04 10:53:59,814 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/daab6abcb1764bfe87a743406f93eacf.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/daab6abcb1764bfe87a743406f93eacf-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/2259719df39a4e0597c03ef49f4c911c.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/2259719df39a4e0597c03ef49f4c911c-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/1210892018ef49139fb716228f4800da.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/1210892018ef49139fb716228f4800da-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/6a9725bd73054bfeb1d17dbed27f1672.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/6a9725bd73054bfeb1d17dbed27f1672-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/cffd4e95da4849b8bfb11d0b9ae14fae.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/cffd4e95da4849b8bfb11d0b9ae14fae-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/599c55505d03422d8dfd8e2522e32fa9.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/599c55505d03422d8dfd8e2522e32fa9-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/8747423eb5444c3cac8b7f007a8c4ba8.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/8747423eb5444c3cac8b7f007a8c4ba8-top] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/.tmp, totalSize=34.9 K 2023-05-04 10:53:59,814 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/1739f54b0b3342b28a862c59b7f3d55a.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/1739f54b0b3342b28a862c59b7f3d55a-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/c90c12d375264697a4271e0e631847f7.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/c90c12d375264697a4271e0e631847f7-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/fa3e0c9057cd41358732e33b9a9e98f7.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/fa3e0c9057cd41358732e33b9a9e98f7-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/575b5478fea041afa8e27fdfaa03ac95.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/575b5478fea041afa8e27fdfaa03ac95-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/2eede810dbbb42ffbb5ade091a470416.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/2eede810dbbb42ffbb5ade091a470416-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/6fee68f6be104910b531998452be1707.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/6fee68f6be104910b531998452be1707-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/bcc61811539a4e10a1ee403686d11536.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/bcc61811539a4e10a1ee403686d11536-top] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/.tmp, totalSize=34.9 K 2023-05-04 10:53:59,814 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting daab6abcb1764bfe87a743406f93eacf.fdafca090ed7cc34c4ad878aac762e9e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683197634044 2023-05-04 10:53:59,814 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting 1739f54b0b3342b28a862c59b7f3d55a.fdafca090ed7cc34c4ad878aac762e9e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683197634044 2023-05-04 10:53:59,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec. 2023-05-04 10:53:59,815 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting 2259719df39a4e0597c03ef49f4c911c.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683197634257 2023-05-04 10:53:59,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec. 2023-05-04 10:53:59,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707. 2023-05-04 10:53:59,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4110ba1eeda845d33e575cf303f91707, NAME => 'testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.', STARTKEY => '', ENDKEY => '41'} 2023-05-04 10:53:59,815 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting c90c12d375264697a4271e0e631847f7.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683197634257 2023-05-04 10:53:59,815 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting 1210892018ef49139fb716228f4800da.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683197634465 2023-05-04 10:53:59,815 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=9db21ffe8cb46d9683ca276a900101ec, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:59,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:53:59,815 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197639815"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197639815"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197639815"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197639815"}]},"ts":"1683197639815"} 2023-05-04 10:53:59,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 4110ba1eeda845d33e575cf303f91707 2023-05-04 10:53:59,816 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting fa3e0c9057cd41358732e33b9a9e98f7.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683197634465 2023-05-04 10:53:59,816 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting 6a9725bd73054bfeb1d17dbed27f1672.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683197634667 2023-05-04 10:53:59,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:53:59,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4110ba1eeda845d33e575cf303f91707 2023-05-04 10:53:59,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4110ba1eeda845d33e575cf303f91707 2023-05-04 10:53:59,816 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting cffd4e95da4849b8bfb11d0b9ae14fae.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1683197634884 2023-05-04 10:53:59,816 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting 575b5478fea041afa8e27fdfaa03ac95.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683197634667 2023-05-04 10:53:59,817 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting 599c55505d03422d8dfd8e2522e32fa9.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1683197635103 2023-05-04 10:53:59,818 INFO [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4110ba1eeda845d33e575cf303f91707 2023-05-04 10:53:59,818 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting 2eede810dbbb42ffbb5ade091a470416.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1683197634884 2023-05-04 10:53:59,818 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting 8747423eb5444c3cac8b7f007a8c4ba8.fdafca090ed7cc34c4ad878aac762e9e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683197635713 2023-05-04 10:53:59,819 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting 6fee68f6be104910b531998452be1707.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1683197635103 2023-05-04 10:53:59,819 DEBUG [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1 2023-05-04 10:53:59,820 DEBUG [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1 2023-05-04 10:53:59,820 INFO [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4110ba1eeda845d33e575cf303f91707 columnFamilyName cf1 2023-05-04 10:53:59,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-05-04 10:53:59,820 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting bcc61811539a4e10a1ee403686d11536.fdafca090ed7cc34c4ad878aac762e9e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683197635713 2023-05-04 10:53:59,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure 9db21ffe8cb46d9683ca276a900101ec, server=jenkins-hbase4.apache.org,42285,1683197625528 in 260 msec 2023-05-04 10:53:59,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=9db21ffe8cb46d9683ca276a900101ec, ASSIGN in 427 msec 2023-05-04 10:53:59,841 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] throttle.PressureAwareThroughputController(145): 9db21ffe8cb46d9683ca276a900101ec#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:53:59,854 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9db21ffe8cb46d9683ca276a900101ec#cf1#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:53:59,861 DEBUG [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/1739f54b0b3342b28a862c59b7f3d55a.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/1739f54b0b3342b28a862c59b7f3d55a-bottom 2023-05-04 10:53:59,867 DEBUG [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/2eede810dbbb42ffbb5ade091a470416.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/2eede810dbbb42ffbb5ade091a470416-bottom 2023-05-04 10:53:59,877 DEBUG [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/575b5478fea041afa8e27fdfaa03ac95.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/575b5478fea041afa8e27fdfaa03ac95-bottom 2023-05-04 10:53:59,883 DEBUG [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/6fee68f6be104910b531998452be1707.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/6fee68f6be104910b531998452be1707-bottom 2023-05-04 10:53:59,888 DEBUG [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/bcc61811539a4e10a1ee403686d11536.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/bcc61811539a4e10a1ee403686d11536-bottom 2023-05-04 10:53:59,888 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/.tmp/cf2/3684d2b83c1842b2a5a8d46f7f1b2fbf as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/3684d2b83c1842b2a5a8d46f7f1b2fbf 2023-05-04 10:53:59,889 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/.tmp/cf1/10394d471654438bb3ec4bad5920999c as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/10394d471654438bb3ec4bad5920999c 2023-05-04 10:53:59,894 DEBUG [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/c90c12d375264697a4271e0e631847f7.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/c90c12d375264697a4271e0e631847f7-bottom 2023-05-04 10:53:59,896 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 9db21ffe8cb46d9683ca276a900101ec/cf2 of 9db21ffe8cb46d9683ca276a900101ec into 3684d2b83c1842b2a5a8d46f7f1b2fbf(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:53:59,896 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 9db21ffe8cb46d9683ca276a900101ec: 2023-05-04 10:53:59,896 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec., storeName=9db21ffe8cb46d9683ca276a900101ec/cf2, priority=9, startTime=1683197639812; duration=0sec 2023-05-04 10:53:59,897 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:53:59,897 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 9db21ffe8cb46d9683ca276a900101ec/cf1 of 9db21ffe8cb46d9683ca276a900101ec into 10394d471654438bb3ec4bad5920999c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:53:59,897 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 9db21ffe8cb46d9683ca276a900101ec: 2023-05-04 10:53:59,897 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec., storeName=9db21ffe8cb46d9683ca276a900101ec/cf1, priority=9, startTime=1683197639811; duration=0sec 2023-05-04 10:53:59,897 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:53:59,899 DEBUG [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/fa3e0c9057cd41358732e33b9a9e98f7.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/fa3e0c9057cd41358732e33b9a9e98f7-bottom 2023-05-04 10:53:59,899 INFO [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] regionserver.HStore(310): Store=4110ba1eeda845d33e575cf303f91707/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:59,899 INFO [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4110ba1eeda845d33e575cf303f91707 2023-05-04 10:53:59,900 DEBUG [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2 2023-05-04 10:53:59,900 DEBUG [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2 2023-05-04 10:53:59,901 INFO [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4110ba1eeda845d33e575cf303f91707 columnFamilyName cf2 2023-05-04 10:53:59,915 DEBUG [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/1210892018ef49139fb716228f4800da.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/1210892018ef49139fb716228f4800da-bottom 2023-05-04 10:53:59,919 DEBUG [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/2259719df39a4e0597c03ef49f4c911c.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/2259719df39a4e0597c03ef49f4c911c-bottom 2023-05-04 10:53:59,923 DEBUG [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/599c55505d03422d8dfd8e2522e32fa9.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/599c55505d03422d8dfd8e2522e32fa9-bottom 2023-05-04 10:53:59,928 DEBUG [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/6a9725bd73054bfeb1d17dbed27f1672.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/6a9725bd73054bfeb1d17dbed27f1672-bottom 2023-05-04 10:53:59,935 DEBUG [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/8747423eb5444c3cac8b7f007a8c4ba8.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/8747423eb5444c3cac8b7f007a8c4ba8-bottom 2023-05-04 10:53:59,939 DEBUG [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/cffd4e95da4849b8bfb11d0b9ae14fae.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/cffd4e95da4849b8bfb11d0b9ae14fae-bottom 2023-05-04 10:53:59,943 DEBUG [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/daab6abcb1764bfe87a743406f93eacf.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/daab6abcb1764bfe87a743406f93eacf-bottom 2023-05-04 10:53:59,943 INFO [StoreOpener-4110ba1eeda845d33e575cf303f91707-1] regionserver.HStore(310): Store=4110ba1eeda845d33e575cf303f91707/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:53:59,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707 2023-05-04 10:53:59,945 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707 2023-05-04 10:53:59,946 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:53:59,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4110ba1eeda845d33e575cf303f91707 2023-05-04 10:53:59,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4110ba1eeda845d33e575cf303f91707; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74683316, jitterRate=0.11286813020706177}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:53:59,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4110ba1eeda845d33e575cf303f91707: 2023-05-04 10:53:59,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707., pid=33, masterSystemTime=1683197639704 2023-05-04 10:53:59,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:53:59,950 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-04 10:53:59,950 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:53:59,950 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:53:59,950 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707. 2023-05-04 10:53:59,950 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1912): 4110ba1eeda845d33e575cf303f91707/cf1 is initiating minor compaction (all files) 2023-05-04 10:53:59,951 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707. 2023-05-04 10:53:59,951 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 4110ba1eeda845d33e575cf303f91707/cf1 in testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707. 2023-05-04 10:53:59,951 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1912): 4110ba1eeda845d33e575cf303f91707/cf2 is initiating minor compaction (all files) 2023-05-04 10:53:59,951 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 4110ba1eeda845d33e575cf303f91707/cf2 in testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707. 2023-05-04 10:53:59,951 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/1739f54b0b3342b28a862c59b7f3d55a.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/1739f54b0b3342b28a862c59b7f3d55a-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/c90c12d375264697a4271e0e631847f7.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/c90c12d375264697a4271e0e631847f7-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/fa3e0c9057cd41358732e33b9a9e98f7.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/fa3e0c9057cd41358732e33b9a9e98f7-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/575b5478fea041afa8e27fdfaa03ac95.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/575b5478fea041afa8e27fdfaa03ac95-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/2eede810dbbb42ffbb5ade091a470416.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/2eede810dbbb42ffbb5ade091a470416-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/6fee68f6be104910b531998452be1707.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/6fee68f6be104910b531998452be1707-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/bcc61811539a4e10a1ee403686d11536.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/bcc61811539a4e10a1ee403686d11536-bottom] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/.tmp, totalSize=34.9 K 2023-05-04 10:53:59,951 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/daab6abcb1764bfe87a743406f93eacf.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/daab6abcb1764bfe87a743406f93eacf-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/2259719df39a4e0597c03ef49f4c911c.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/2259719df39a4e0597c03ef49f4c911c-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/1210892018ef49139fb716228f4800da.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/1210892018ef49139fb716228f4800da-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/6a9725bd73054bfeb1d17dbed27f1672.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/6a9725bd73054bfeb1d17dbed27f1672-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/cffd4e95da4849b8bfb11d0b9ae14fae.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/cffd4e95da4849b8bfb11d0b9ae14fae-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/599c55505d03422d8dfd8e2522e32fa9.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/599c55505d03422d8dfd8e2522e32fa9-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/8747423eb5444c3cac8b7f007a8c4ba8.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/8747423eb5444c3cac8b7f007a8c4ba8-bottom] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/.tmp, totalSize=34.9 K 2023-05-04 10:53:59,952 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting 1739f54b0b3342b28a862c59b7f3d55a.fdafca090ed7cc34c4ad878aac762e9e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683197634044 2023-05-04 10:53:59,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707. 2023-05-04 10:53:59,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707. 2023-05-04 10:53:59,952 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting daab6abcb1764bfe87a743406f93eacf.fdafca090ed7cc34c4ad878aac762e9e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683197634044 2023-05-04 10:53:59,952 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting c90c12d375264697a4271e0e631847f7.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683197634257 2023-05-04 10:53:59,953 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting 2259719df39a4e0597c03ef49f4c911c.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683197634257 2023-05-04 10:53:59,953 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=4110ba1eeda845d33e575cf303f91707, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:53:59,953 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting fa3e0c9057cd41358732e33b9a9e98f7.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683197634465 2023-05-04 10:53:59,953 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197639953"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197639953"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197639953"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197639953"}]},"ts":"1683197639953"} 2023-05-04 10:53:59,953 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting 1210892018ef49139fb716228f4800da.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683197634465 2023-05-04 10:53:59,953 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting 575b5478fea041afa8e27fdfaa03ac95.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683197634667 2023-05-04 10:53:59,954 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting 6a9725bd73054bfeb1d17dbed27f1672.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683197634667 2023-05-04 10:53:59,954 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting 2eede810dbbb42ffbb5ade091a470416.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683197634884 2023-05-04 10:53:59,954 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting cffd4e95da4849b8bfb11d0b9ae14fae.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683197634884 2023-05-04 10:53:59,954 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting 6fee68f6be104910b531998452be1707.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683197635103 2023-05-04 10:53:59,955 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting 599c55505d03422d8dfd8e2522e32fa9.fdafca090ed7cc34c4ad878aac762e9e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683197635103 2023-05-04 10:53:59,955 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting bcc61811539a4e10a1ee403686d11536.fdafca090ed7cc34c4ad878aac762e9e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683197635713 2023-05-04 10:53:59,955 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting 8747423eb5444c3cac8b7f007a8c4ba8.fdafca090ed7cc34c4ad878aac762e9e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683197635713 2023-05-04 10:53:59,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-05-04 10:53:59,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure 4110ba1eeda845d33e575cf303f91707, server=jenkins-hbase4.apache.org,42285,1683197625528 in 402 msec 2023-05-04 10:53:59,960 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-05-04 10:53:59,960 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4110ba1eeda845d33e575cf303f91707, ASSIGN in 563 msec 2023-05-04 10:53:59,962 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec in 3.9830 sec 2023-05-04 10:53:59,971 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] throttle.PressureAwareThroughputController(145): 4110ba1eeda845d33e575cf303f91707#cf2#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:53:59,973 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4110ba1eeda845d33e575cf303f91707#cf1#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:53:59,997 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/.tmp/cf2/211e705ae1e24e2cb06aff15b71576c3 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/211e705ae1e24e2cb06aff15b71576c3 2023-05-04 10:53:59,997 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/.tmp/cf1/d364f8d4a5b74e8a896c8fb1b853b966 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/d364f8d4a5b74e8a896c8fb1b853b966 2023-05-04 10:54:00,003 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4110ba1eeda845d33e575cf303f91707/cf2 of 4110ba1eeda845d33e575cf303f91707 into 211e705ae1e24e2cb06aff15b71576c3(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:54:00,003 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4110ba1eeda845d33e575cf303f91707/cf1 of 4110ba1eeda845d33e575cf303f91707 into d364f8d4a5b74e8a896c8fb1b853b966(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:54:00,003 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 4110ba1eeda845d33e575cf303f91707: 2023-05-04 10:54:00,003 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 4110ba1eeda845d33e575cf303f91707: 2023-05-04 10:54:00,003 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707., storeName=4110ba1eeda845d33e575cf303f91707/cf2, priority=9, startTime=1683197639950; duration=0sec 2023-05-04 10:54:00,003 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707., storeName=4110ba1eeda845d33e575cf303f91707/cf1, priority=9, startTime=1683197639949; duration=0sec 2023-05-04 10:54:00,003 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:54:00,003 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:54:00,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42285] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:37016 deadline: 1683197700143, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e. is not online on jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:00,256 DEBUG [Listener at localhost/35483] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-04 10:54:00,264 INFO [Listener at localhost/35483] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-05-04 10:54:00,267 DEBUG [Listener at localhost/35483] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-04 10:54:00,283 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:54:00,285 INFO [Listener at localhost/35483] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-05-04 10:54:00,286 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-05-04 10:54:00,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:00,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-05-04 10:54:00,290 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197640290"}]},"ts":"1683197640290"} 2023-05-04 10:54:00,291 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-05-04 10:54:00,293 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-05-04 10:54:00,294 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4110ba1eeda845d33e575cf303f91707, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=9db21ffe8cb46d9683ca276a900101ec, UNASSIGN}] 2023-05-04 10:54:00,295 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=9db21ffe8cb46d9683ca276a900101ec, UNASSIGN 2023-05-04 10:54:00,295 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=9db21ffe8cb46d9683ca276a900101ec, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:00,295 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197640295"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197640295"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197640295"}]},"ts":"1683197640295"} 2023-05-04 10:54:00,297 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 9db21ffe8cb46d9683ca276a900101ec, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:54:00,299 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4110ba1eeda845d33e575cf303f91707, UNASSIGN 2023-05-04 10:54:00,300 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=4110ba1eeda845d33e575cf303f91707, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:00,300 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197640300"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197640300"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197640300"}]},"ts":"1683197640300"} 2023-05-04 10:54:00,302 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 4110ba1eeda845d33e575cf303f91707, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:54:00,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-05-04 10:54:00,449 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 4110ba1eeda845d33e575cf303f91707 2023-05-04 10:54:00,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4110ba1eeda845d33e575cf303f91707, disabling compactions & flushes 2023-05-04 10:54:00,449 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707. 2023-05-04 10:54:00,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707. 2023-05-04 10:54:00,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707. after waiting 0 ms 2023-05-04 10:54:00,450 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707. 2023-05-04 10:54:00,463 DEBUG [StoreCloser-testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/1739f54b0b3342b28a862c59b7f3d55a.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/1739f54b0b3342b28a862c59b7f3d55a-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/c90c12d375264697a4271e0e631847f7.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/c90c12d375264697a4271e0e631847f7-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/fa3e0c9057cd41358732e33b9a9e98f7.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/fa3e0c9057cd41358732e33b9a9e98f7-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/575b5478fea041afa8e27fdfaa03ac95.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/575b5478fea041afa8e27fdfaa03ac95-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/2eede810dbbb42ffbb5ade091a470416.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/2eede810dbbb42ffbb5ade091a470416-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/6fee68f6be104910b531998452be1707.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/6fee68f6be104910b531998452be1707-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/bcc61811539a4e10a1ee403686d11536.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/bcc61811539a4e10a1ee403686d11536-bottom] to archive 2023-05-04 10:54:00,464 DEBUG [StoreCloser-testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:54:00,466 DEBUG [StoreCloser-testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/1739f54b0b3342b28a862c59b7f3d55a.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/1739f54b0b3342b28a862c59b7f3d55a.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,467 DEBUG [StoreCloser-testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/c90c12d375264697a4271e0e631847f7.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/c90c12d375264697a4271e0e631847f7.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,469 DEBUG [StoreCloser-testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/fa3e0c9057cd41358732e33b9a9e98f7.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/fa3e0c9057cd41358732e33b9a9e98f7.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,470 DEBUG [StoreCloser-testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/575b5478fea041afa8e27fdfaa03ac95.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/575b5478fea041afa8e27fdfaa03ac95.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,471 DEBUG [StoreCloser-testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/2eede810dbbb42ffbb5ade091a470416.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/2eede810dbbb42ffbb5ade091a470416.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,472 DEBUG [StoreCloser-testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/6fee68f6be104910b531998452be1707.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/6fee68f6be104910b531998452be1707.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,474 DEBUG [StoreCloser-testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/bcc61811539a4e10a1ee403686d11536.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/bcc61811539a4e10a1ee403686d11536.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,475 DEBUG [StoreCloser-testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/daab6abcb1764bfe87a743406f93eacf.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/daab6abcb1764bfe87a743406f93eacf-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/2259719df39a4e0597c03ef49f4c911c.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/2259719df39a4e0597c03ef49f4c911c-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/1210892018ef49139fb716228f4800da.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/1210892018ef49139fb716228f4800da-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/6a9725bd73054bfeb1d17dbed27f1672.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/6a9725bd73054bfeb1d17dbed27f1672-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/cffd4e95da4849b8bfb11d0b9ae14fae.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/cffd4e95da4849b8bfb11d0b9ae14fae-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/599c55505d03422d8dfd8e2522e32fa9.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/599c55505d03422d8dfd8e2522e32fa9-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/8747423eb5444c3cac8b7f007a8c4ba8.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/8747423eb5444c3cac8b7f007a8c4ba8-bottom] to archive 2023-05-04 10:54:00,476 DEBUG [StoreCloser-testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:54:00,478 DEBUG [StoreCloser-testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/daab6abcb1764bfe87a743406f93eacf.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/daab6abcb1764bfe87a743406f93eacf.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,479 DEBUG [StoreCloser-testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/2259719df39a4e0597c03ef49f4c911c.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/2259719df39a4e0597c03ef49f4c911c.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,480 DEBUG [StoreCloser-testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/1210892018ef49139fb716228f4800da.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/1210892018ef49139fb716228f4800da.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,482 DEBUG [StoreCloser-testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/6a9725bd73054bfeb1d17dbed27f1672.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/6a9725bd73054bfeb1d17dbed27f1672.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,483 DEBUG [StoreCloser-testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/cffd4e95da4849b8bfb11d0b9ae14fae.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/cffd4e95da4849b8bfb11d0b9ae14fae.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,485 DEBUG [StoreCloser-testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/599c55505d03422d8dfd8e2522e32fa9.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/599c55505d03422d8dfd8e2522e32fa9.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,486 DEBUG [StoreCloser-testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/8747423eb5444c3cac8b7f007a8c4ba8.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/8747423eb5444c3cac8b7f007a8c4ba8.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,490 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-04 10:54:00,491 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:54:00,491 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707. 2023-05-04 10:54:00,491 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4110ba1eeda845d33e575cf303f91707: 2023-05-04 10:54:00,493 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 4110ba1eeda845d33e575cf303f91707 2023-05-04 10:54:00,493 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:00,493 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 9db21ffe8cb46d9683ca276a900101ec, disabling compactions & flushes 2023-05-04 10:54:00,493 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec. 2023-05-04 10:54:00,493 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec. 2023-05-04 10:54:00,493 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec. after waiting 0 ms 2023-05-04 10:54:00,493 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec. 2023-05-04 10:54:00,493 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=4110ba1eeda845d33e575cf303f91707, regionState=CLOSED 2023-05-04 10:54:00,493 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197640493"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197640493"}]},"ts":"1683197640493"} 2023-05-04 10:54:00,495 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/1739f54b0b3342b28a862c59b7f3d55a.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/1739f54b0b3342b28a862c59b7f3d55a-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/c90c12d375264697a4271e0e631847f7.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/c90c12d375264697a4271e0e631847f7-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/fa3e0c9057cd41358732e33b9a9e98f7.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/fa3e0c9057cd41358732e33b9a9e98f7-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/575b5478fea041afa8e27fdfaa03ac95.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/575b5478fea041afa8e27fdfaa03ac95-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/2eede810dbbb42ffbb5ade091a470416.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/2eede810dbbb42ffbb5ade091a470416-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/6fee68f6be104910b531998452be1707.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/6fee68f6be104910b531998452be1707-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/bcc61811539a4e10a1ee403686d11536.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/bcc61811539a4e10a1ee403686d11536-top] to archive 2023-05-04 10:54:00,496 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:54:00,498 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/1739f54b0b3342b28a862c59b7f3d55a.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/1739f54b0b3342b28a862c59b7f3d55a.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-05-04 10:54:00,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 4110ba1eeda845d33e575cf303f91707, server=jenkins-hbase4.apache.org,42285,1683197625528 in 194 msec 2023-05-04 10:54:00,500 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/c90c12d375264697a4271e0e631847f7.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/c90c12d375264697a4271e0e631847f7.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,500 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4110ba1eeda845d33e575cf303f91707, UNASSIGN in 204 msec 2023-05-04 10:54:00,501 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/fa3e0c9057cd41358732e33b9a9e98f7.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/fa3e0c9057cd41358732e33b9a9e98f7.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,502 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/575b5478fea041afa8e27fdfaa03ac95.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/575b5478fea041afa8e27fdfaa03ac95.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,504 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/2eede810dbbb42ffbb5ade091a470416.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/2eede810dbbb42ffbb5ade091a470416.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,505 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/6fee68f6be104910b531998452be1707.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/6fee68f6be104910b531998452be1707.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,507 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/bcc61811539a4e10a1ee403686d11536.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/bcc61811539a4e10a1ee403686d11536.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,509 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/daab6abcb1764bfe87a743406f93eacf.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/daab6abcb1764bfe87a743406f93eacf-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/2259719df39a4e0597c03ef49f4c911c.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/2259719df39a4e0597c03ef49f4c911c-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/1210892018ef49139fb716228f4800da.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/1210892018ef49139fb716228f4800da-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/6a9725bd73054bfeb1d17dbed27f1672.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/6a9725bd73054bfeb1d17dbed27f1672-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/cffd4e95da4849b8bfb11d0b9ae14fae.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/cffd4e95da4849b8bfb11d0b9ae14fae-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/599c55505d03422d8dfd8e2522e32fa9.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/599c55505d03422d8dfd8e2522e32fa9-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/8747423eb5444c3cac8b7f007a8c4ba8.fdafca090ed7cc34c4ad878aac762e9e->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/8747423eb5444c3cac8b7f007a8c4ba8-top] to archive 2023-05-04 10:54:00,509 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:54:00,511 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/daab6abcb1764bfe87a743406f93eacf.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/daab6abcb1764bfe87a743406f93eacf.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,512 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/2259719df39a4e0597c03ef49f4c911c.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/2259719df39a4e0597c03ef49f4c911c.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,513 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/1210892018ef49139fb716228f4800da.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/1210892018ef49139fb716228f4800da.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,514 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/6a9725bd73054bfeb1d17dbed27f1672.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/6a9725bd73054bfeb1d17dbed27f1672.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,515 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/cffd4e95da4849b8bfb11d0b9ae14fae.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/cffd4e95da4849b8bfb11d0b9ae14fae.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,517 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/599c55505d03422d8dfd8e2522e32fa9.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/599c55505d03422d8dfd8e2522e32fa9.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,518 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/8747423eb5444c3cac8b7f007a8c4ba8.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/8747423eb5444c3cac8b7f007a8c4ba8.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,522 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-04 10:54:00,522 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:54:00,522 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec. 2023-05-04 10:54:00,522 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 9db21ffe8cb46d9683ca276a900101ec: 2023-05-04 10:54:00,524 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:00,525 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=9db21ffe8cb46d9683ca276a900101ec, regionState=CLOSED 2023-05-04 10:54:00,525 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197640524"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197640524"}]},"ts":"1683197640524"} 2023-05-04 10:54:00,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-05-04 10:54:00,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 9db21ffe8cb46d9683ca276a900101ec, server=jenkins-hbase4.apache.org,42285,1683197625528 in 229 msec 2023-05-04 10:54:00,530 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=35 2023-05-04 10:54:00,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=9db21ffe8cb46d9683ca276a900101ec, UNASSIGN in 235 msec 2023-05-04 10:54:00,531 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197640531"}]},"ts":"1683197640531"} 2023-05-04 10:54:00,532 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-05-04 10:54:00,534 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-05-04 10:54:00,536 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 249 msec 2023-05-04 10:54:00,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-05-04 10:54:00,593 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-05-04 10:54:00,594 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-05-04 10:54:00,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:00,597 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:00,598 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:00,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-05-04 10:54:00,605 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:00,605 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707 2023-05-04 10:54:00,605 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,607 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/recovered.edits] 2023-05-04 10:54:00,608 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/recovered.edits] 2023-05-04 10:54:00,608 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/recovered.edits] 2023-05-04 10:54:00,612 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/.regioninfo to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/.regioninfo 2023-05-04 10:54:00,612 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/.regioninfo to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/.regioninfo 2023-05-04 10:54:00,612 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/1739f54b0b3342b28a862c59b7f3d55a to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/1739f54b0b3342b28a862c59b7f3d55a 2023-05-04 10:54:00,614 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/2eede810dbbb42ffbb5ade091a470416 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/2eede810dbbb42ffbb5ade091a470416 2023-05-04 10:54:00,615 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/575b5478fea041afa8e27fdfaa03ac95 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/575b5478fea041afa8e27fdfaa03ac95 2023-05-04 10:54:00,616 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf1/1739f54b0b3342b28a862c59b7f3d55a.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf1/1739f54b0b3342b28a862c59b7f3d55a.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,616 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf1/1739f54b0b3342b28a862c59b7f3d55a.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf1/1739f54b0b3342b28a862c59b7f3d55a.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,617 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/6fee68f6be104910b531998452be1707 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/6fee68f6be104910b531998452be1707 2023-05-04 10:54:00,617 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf1/2eede810dbbb42ffbb5ade091a470416.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf1/2eede810dbbb42ffbb5ade091a470416.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,617 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf1/2eede810dbbb42ffbb5ade091a470416.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf1/2eede810dbbb42ffbb5ade091a470416.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,619 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf1/575b5478fea041afa8e27fdfaa03ac95.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf1/575b5478fea041afa8e27fdfaa03ac95.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,619 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/bcc61811539a4e10a1ee403686d11536 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/bcc61811539a4e10a1ee403686d11536 2023-05-04 10:54:00,619 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf1/575b5478fea041afa8e27fdfaa03ac95.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf1/575b5478fea041afa8e27fdfaa03ac95.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,621 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf1/6fee68f6be104910b531998452be1707.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf1/6fee68f6be104910b531998452be1707.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,621 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/c90c12d375264697a4271e0e631847f7 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/c90c12d375264697a4271e0e631847f7 2023-05-04 10:54:00,621 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf1/6fee68f6be104910b531998452be1707.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf1/6fee68f6be104910b531998452be1707.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,622 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf1/bcc61811539a4e10a1ee403686d11536.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf1/bcc61811539a4e10a1ee403686d11536.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,623 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/fa3e0c9057cd41358732e33b9a9e98f7 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf1/fa3e0c9057cd41358732e33b9a9e98f7 2023-05-04 10:54:00,623 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf1/bcc61811539a4e10a1ee403686d11536.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf1/bcc61811539a4e10a1ee403686d11536.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,624 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf1/c90c12d375264697a4271e0e631847f7.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf1/c90c12d375264697a4271e0e631847f7.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,625 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf1/c90c12d375264697a4271e0e631847f7.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf1/c90c12d375264697a4271e0e631847f7.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,627 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf1/fa3e0c9057cd41358732e33b9a9e98f7.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf1/fa3e0c9057cd41358732e33b9a9e98f7.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,628 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf1/fa3e0c9057cd41358732e33b9a9e98f7.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf1/fa3e0c9057cd41358732e33b9a9e98f7.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,628 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/1210892018ef49139fb716228f4800da to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/1210892018ef49139fb716228f4800da 2023-05-04 10:54:00,631 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf2/1210892018ef49139fb716228f4800da.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf2/1210892018ef49139fb716228f4800da.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,634 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf2/1210892018ef49139fb716228f4800da.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf2/1210892018ef49139fb716228f4800da.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,634 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/2259719df39a4e0597c03ef49f4c911c to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/2259719df39a4e0597c03ef49f4c911c 2023-05-04 10:54:00,635 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf2/2259719df39a4e0597c03ef49f4c911c.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf2/2259719df39a4e0597c03ef49f4c911c.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,636 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf2/2259719df39a4e0597c03ef49f4c911c.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf2/2259719df39a4e0597c03ef49f4c911c.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,636 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/599c55505d03422d8dfd8e2522e32fa9 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/599c55505d03422d8dfd8e2522e32fa9 2023-05-04 10:54:00,637 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf2/599c55505d03422d8dfd8e2522e32fa9.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf2/599c55505d03422d8dfd8e2522e32fa9.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,638 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf2/599c55505d03422d8dfd8e2522e32fa9.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf2/599c55505d03422d8dfd8e2522e32fa9.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,638 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/6a9725bd73054bfeb1d17dbed27f1672 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/6a9725bd73054bfeb1d17dbed27f1672 2023-05-04 10:54:00,639 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf2/6a9725bd73054bfeb1d17dbed27f1672.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf2/6a9725bd73054bfeb1d17dbed27f1672.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,640 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf2/6a9725bd73054bfeb1d17dbed27f1672.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf2/6a9725bd73054bfeb1d17dbed27f1672.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,640 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/8747423eb5444c3cac8b7f007a8c4ba8 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/8747423eb5444c3cac8b7f007a8c4ba8 2023-05-04 10:54:00,641 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf2/8747423eb5444c3cac8b7f007a8c4ba8.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf2/8747423eb5444c3cac8b7f007a8c4ba8.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,642 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf2/8747423eb5444c3cac8b7f007a8c4ba8.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf2/8747423eb5444c3cac8b7f007a8c4ba8.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,643 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf2/cffd4e95da4849b8bfb11d0b9ae14fae.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf2/cffd4e95da4849b8bfb11d0b9ae14fae.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,643 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/cffd4e95da4849b8bfb11d0b9ae14fae to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/cffd4e95da4849b8bfb11d0b9ae14fae 2023-05-04 10:54:00,644 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf2/cffd4e95da4849b8bfb11d0b9ae14fae.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf2/cffd4e95da4849b8bfb11d0b9ae14fae.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,645 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf2/daab6abcb1764bfe87a743406f93eacf.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/9db21ffe8cb46d9683ca276a900101ec/cf2/daab6abcb1764bfe87a743406f93eacf.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,645 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/daab6abcb1764bfe87a743406f93eacf to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/cf2/daab6abcb1764bfe87a743406f93eacf 2023-05-04 10:54:00,646 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf2/daab6abcb1764bfe87a743406f93eacf.fdafca090ed7cc34c4ad878aac762e9e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/4110ba1eeda845d33e575cf303f91707/cf2/daab6abcb1764bfe87a743406f93eacf.fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,649 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/10394d471654438bb3ec4bad5920999c to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf1/10394d471654438bb3ec4bad5920999c 2023-05-04 10:54:00,650 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/d364f8d4a5b74e8a896c8fb1b853b966 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf1/d364f8d4a5b74e8a896c8fb1b853b966 2023-05-04 10:54:00,650 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/recovered.edits/85.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e/recovered.edits/85.seqid 2023-05-04 10:54:00,651 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/fdafca090ed7cc34c4ad878aac762e9e 2023-05-04 10:54:00,653 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/3684d2b83c1842b2a5a8d46f7f1b2fbf to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/cf2/3684d2b83c1842b2a5a8d46f7f1b2fbf 2023-05-04 10:54:00,653 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/211e705ae1e24e2cb06aff15b71576c3 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/cf2/211e705ae1e24e2cb06aff15b71576c3 2023-05-04 10:54:00,656 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/recovered.edits/90.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec/recovered.edits/90.seqid 2023-05-04 10:54:00,660 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/recovered.edits/90.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707/recovered.edits/90.seqid 2023-05-04 10:54:00,661 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:00,661 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitWithoutPONR/4110ba1eeda845d33e575cf303f91707 2023-05-04 10:54:00,661 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-05-04 10:54:00,664 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:00,667 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-05-04 10:54:00,670 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-05-04 10:54:00,671 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:00,671 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-05-04 10:54:00,671 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197640671"}]},"ts":"9223372036854775807"} 2023-05-04 10:54:00,672 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197640671"}]},"ts":"9223372036854775807"} 2023-05-04 10:54:00,672 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197640671"}]},"ts":"9223372036854775807"} 2023-05-04 10:54:00,674 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-04 10:54:00,674 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => fdafca090ed7cc34c4ad878aac762e9e, NAME => 'testSplitWithoutPONR,,1683197633537.fdafca090ed7cc34c4ad878aac762e9e.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 4110ba1eeda845d33e575cf303f91707, NAME => 'testSplitWithoutPONR,,1683197635977.4110ba1eeda845d33e575cf303f91707.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 9db21ffe8cb46d9683ca276a900101ec, NAME => 'testSplitWithoutPONR,41,1683197635977.9db21ffe8cb46d9683ca276a900101ec.', STARTKEY => '41', ENDKEY => ''}] 2023-05-04 10:54:00,674 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-05-04 10:54:00,674 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683197640674"}]},"ts":"9223372036854775807"} 2023-05-04 10:54:00,676 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-05-04 10:54:00,678 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:00,680 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 84 msec 2023-05-04 10:54:00,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-05-04 10:54:00,700 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-05-04 10:54:00,716 INFO [Listener at localhost/35483] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=403 (was 393) Potentially hanging thread: hconnection-0x2d8012cd-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:56854 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:47172 [Waiting for operation #43] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x451b64de-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/cluster_3e5b0dbc-215f-67e3-28fd-7825f44cb1a4/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:56848 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:54836 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:33222 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/cluster_3e5b0dbc-215f-67e3-28fd-7825f44cb1a4/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:42285-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:55016 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:54802 [Waiting for operation #29] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:47112 [Waiting for operation #34] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:58528 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:59082 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:58450 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/cluster_3e5b0dbc-215f-67e3-28fd-7825f44cb1a4/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/cluster_3e5b0dbc-215f-67e3-28fd-7825f44cb1a4/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:44818 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:58420 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:54752 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:47410 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x451b64de-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:33304 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=847 (was 846) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=340 (was 384), ProcessCount=170 (was 170), AvailableMemoryMB=6540 (was 6536) - AvailableMemoryMB LEAK? - 2023-05-04 10:54:00,732 INFO [Listener at localhost/35483] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=403, OpenFileDescriptor=847, MaxFileDescriptor=60000, SystemLoadAverage=340, ProcessCount=170, AvailableMemoryMB=6539 2023-05-04 10:54:00,732 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:54:00,735 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-04 10:54:00,736 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-04 10:54:00,737 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:00,739 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-05-04 10:54:00,740 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-04 10:54:00,742 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:00,743 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369 empty. 2023-05-04 10:54:00,743 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:00,743 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-05-04 10:54:00,756 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-05-04 10:54:00,757 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => ff484ddaba859c44b9223e0332036369, NAME => 'testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp 2023-05-04 10:54:00,768 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:54:00,768 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing ff484ddaba859c44b9223e0332036369, disabling compactions & flushes 2023-05-04 10:54:00,768 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:00,768 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:00,768 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. after waiting 0 ms 2023-05-04 10:54:00,768 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:00,768 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:00,768 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for ff484ddaba859c44b9223e0332036369: 2023-05-04 10:54:00,771 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-05-04 10:54:00,772 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197640772"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197640772"}]},"ts":"1683197640772"} 2023-05-04 10:54:00,774 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-04 10:54:00,775 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-04 10:54:00,775 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197640775"}]},"ts":"1683197640775"} 2023-05-04 10:54:00,776 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-05-04 10:54:00,779 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-04 10:54:00,780 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-04 10:54:00,780 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-04 10:54:00,780 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-04 10:54:00,780 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-04 10:54:00,780 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN}] 2023-05-04 10:54:00,781 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:54:00,782 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37343,1683197625438; forceNewPlan=false, retain=false 2023-05-04 10:54:00,932 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-04 10:54:00,934 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=ff484ddaba859c44b9223e0332036369, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:00,934 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197640934"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197640934"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197640934"}]},"ts":"1683197640934"} 2023-05-04 10:54:00,936 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438}] 2023-05-04 10:54:01,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:01,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ff484ddaba859c44b9223e0332036369, NAME => 'testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.', STARTKEY => '', ENDKEY => ''} 2023-05-04 10:54:01,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:54:01,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:01,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:54:01,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:01,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:01,097 INFO [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:01,099 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1 2023-05-04 10:54:01,099 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1 2023-05-04 10:54:01,099 INFO [StoreOpener-ff484ddaba859c44b9223e0332036369-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ff484ddaba859c44b9223e0332036369 columnFamilyName cf1 2023-05-04 10:54:01,100 INFO [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(310): Store=ff484ddaba859c44b9223e0332036369/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:54:01,100 INFO [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:01,102 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2 2023-05-04 10:54:01,102 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2 2023-05-04 10:54:01,102 INFO [StoreOpener-ff484ddaba859c44b9223e0332036369-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ff484ddaba859c44b9223e0332036369 columnFamilyName cf2 2023-05-04 10:54:01,103 INFO [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(310): Store=ff484ddaba859c44b9223e0332036369/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:54:01,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:01,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:01,106 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:54:01,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:01,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-04 10:54:01,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ff484ddaba859c44b9223e0332036369; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64855072, jitterRate=-0.0335841178894043}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:54:01,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ff484ddaba859c44b9223e0332036369: 2023-05-04 10:54:01,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369., pid=43, masterSystemTime=1683197641088 2023-05-04 10:54:01,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:01,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:01,112 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=ff484ddaba859c44b9223e0332036369, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,112 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197641112"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197641112"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197641112"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197641112"}]},"ts":"1683197641112"} 2023-05-04 10:54:01,116 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-05-04 10:54:01,116 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 in 178 msec 2023-05-04 10:54:01,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-05-04 10:54:01,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN in 336 msec 2023-05-04 10:54:01,119 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-04 10:54:01,119 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197641119"}]},"ts":"1683197641119"} 2023-05-04 10:54:01,120 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-05-04 10:54:01,126 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-05-04 10:54:01,128 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 390 msec 2023-05-04 10:54:01,241 DEBUG [Listener at localhost/35483] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-04 10:54:01,244 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39976, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-04 10:54:01,253 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:54:01,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-04 10:54:01,255 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-04 10:54:01,255 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:01,255 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-04 10:54:01,255 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-04 10:54:01,255 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,255 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:01,257 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:01,257 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,257 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:01,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:01,257 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:01,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:01,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:01,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:01,257 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:01,257 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:54:01,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:01,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:01,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:01,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:01,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:01,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:01,258 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-04 10:54:01,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:01,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:01,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:01,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:01,258 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-04 10:54:01,258 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-04 10:54:01,258 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-04 10:54:01,259 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:01,258 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-04 10:54:01,259 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-04 10:54:01,259 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:54:01,259 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-04 10:54:01,259 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-04 10:54:01,259 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-04 10:54:01,259 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:01,259 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-04 10:54:01,259 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:54:01,259 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-04 10:54:01,259 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:01,259 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:54:01,259 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:54:01,259 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:54:01,259 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-04 10:54:01,259 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:01,259 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-04 10:54:01,259 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. started... 2023-05-04 10:54:01,261 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ff484ddaba859c44b9223e0332036369 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-04 10:54:01,262 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,265 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,266 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,266 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:01,268 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,268 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:01,281 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf1/81ce75c883534f40a8d1da89d3df7e7c 2023-05-04 10:54:01,300 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf2/75b40abe85aa4e2fafa1aac676320bf1 2023-05-04 10:54:01,306 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf1/81ce75c883534f40a8d1da89d3df7e7c as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/81ce75c883534f40a8d1da89d3df7e7c 2023-05-04 10:54:01,311 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/81ce75c883534f40a8d1da89d3df7e7c, entries=2, sequenceid=6, filesize=4.8 K 2023-05-04 10:54:01,312 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf2/75b40abe85aa4e2fafa1aac676320bf1 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/75b40abe85aa4e2fafa1aac676320bf1 2023-05-04 10:54:01,317 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/75b40abe85aa4e2fafa1aac676320bf1, entries=2, sequenceid=6, filesize=4.8 K 2023-05-04 10:54:01,318 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ff484ddaba859c44b9223e0332036369 in 57ms, sequenceid=6, compaction requested=false 2023-05-04 10:54:01,318 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-05-04 10:54:01,318 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ff484ddaba859c44b9223e0332036369: 2023-05-04 10:54:01,318 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:01,318 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:54:01,319 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:54:01,319 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,319 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-04 10:54:01,319 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-04 10:54:01,320 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,320 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:01,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:01,321 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,321 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:01,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:01,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:01,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:01,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:01,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:01,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:01,323 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-04 10:54:01,323 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-04 10:54:01,323 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ca4af40[Count = 0] remaining members to acquire global barrier 2023-05-04 10:54:01,323 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,324 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,324 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,324 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,325 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:54:01,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,325 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-04 10:54:01,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,325 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:54:01,325 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,325 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:54:01,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,325 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:54:01,325 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:54:01,326 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-04 10:54:01,325 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-04 10:54:01,326 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:54:01,326 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:54:01,326 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-04 10:54:01,327 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:01,327 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-04 10:54:01,329 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,329 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-04 10:54:01,329 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:01,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:01,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:01,329 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-04 10:54:01,330 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-04 10:54:01,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:01,330 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:01,330 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-04 10:54:01,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:01,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:01,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:01,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:01,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:01,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:01,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:01,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:01,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,37343,1683197625438': 2023-05-04 10:54:01,334 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-04 10:54:01,334 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-04 10:54:01,334 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:54:01,334 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:54:01,334 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-04 10:54:01,334 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:54:01,336 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,336 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,336 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:01,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:01,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,336 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:01,336 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,336 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:01,336 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:01,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:01,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,337 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:01,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:01,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:01,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:01,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:01,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:01,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:01,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:01,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,338 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:01,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,338 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:01,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:01,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:01,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:01,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:01,341 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:01,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:01,341 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:01,341 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:01,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:01,344 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:01,344 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:01,344 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:01,344 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:01,344 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:01,344 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,344 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:01,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:01,344 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:01,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:01,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:01,344 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-04 10:54:01,345 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:54:01,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:01,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:01,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:01,344 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:01,344 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:01,344 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:01,345 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:01,345 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:01,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:01,345 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:01,345 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-04 10:54:01,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:01,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:01,345 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:54:01,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:01,345 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:01,346 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:01,346 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,346 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,446 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:54:01,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:54:01,465 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:54:01,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-04 10:54:01,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-04 10:54:01,467 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-04 10:54:01,467 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:01,468 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-04 10:54:01,468 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-04 10:54:01,468 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,468 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:01,470 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:01,470 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:01,470 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:01,470 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:01,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:01,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:01,470 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,470 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:54:01,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:01,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:01,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:01,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:01,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:01,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:01,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:01,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:01,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:01,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:01,472 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-04 10:54:01,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:01,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:01,472 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-04 10:54:01,472 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-04 10:54:01,472 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-04 10:54:01,472 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-04 10:54:01,472 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-04 10:54:01,473 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:01,473 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:54:01,473 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-04 10:54:01,473 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-04 10:54:01,475 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,475 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,475 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:01,480 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-04 10:54:01,480 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-04 10:54:01,481 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-04 10:54:01,481 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-04 10:54:01,481 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-04 10:54:01,481 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-04 10:54:01,481 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:01,481 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:01,481 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:54:01,481 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:54:01,481 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-04 10:54:01,481 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-04 10:54:01,481 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:54:01,481 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:54:01,482 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:01,482 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. started... 2023-05-04 10:54:01,482 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ff484ddaba859c44b9223e0332036369 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:54:01,483 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,483 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,483 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:01,498 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf1/de0e333521134ffb8370a273ad000f44 2023-05-04 10:54:01,515 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf2/ea0f76018bc64a8383fdbba25fa2a396 2023-05-04 10:54:01,520 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf1/de0e333521134ffb8370a273ad000f44 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/de0e333521134ffb8370a273ad000f44 2023-05-04 10:54:01,526 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/de0e333521134ffb8370a273ad000f44, entries=10, sequenceid=19, filesize=5.0 K 2023-05-04 10:54:01,527 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf2/ea0f76018bc64a8383fdbba25fa2a396 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/ea0f76018bc64a8383fdbba25fa2a396 2023-05-04 10:54:01,531 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/ea0f76018bc64a8383fdbba25fa2a396, entries=10, sequenceid=19, filesize=5.0 K 2023-05-04 10:54:01,532 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ff484ddaba859c44b9223e0332036369 in 50ms, sequenceid=19, compaction requested=false 2023-05-04 10:54:01,533 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ff484ddaba859c44b9223e0332036369: 2023-05-04 10:54:01,533 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:01,533 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:54:01,533 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:54:01,533 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,533 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-04 10:54:01,533 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-04 10:54:01,591 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,591 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:01,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:01,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:01,592 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,592 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:01,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:01,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:01,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:01,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:01,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:01,594 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-04 10:54:01,595 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@26fd9488[Count = 0] remaining members to acquire global barrier 2023-05-04 10:54:01,595 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-04 10:54:01,595 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,676 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,676 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,676 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,676 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:54:01,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,677 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-04 10:54:01,676 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,677 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:54:01,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,677 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:54:01,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,677 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:54:01,677 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:54:01,677 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-04 10:54:01,677 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-04 10:54:01,677 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:54:01,677 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:54:01,739 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-04 10:54:01,739 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:01,739 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-04 10:54:01,876 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-04 10:54:01,876 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,876 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:01,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:01,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:01,876 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-04 10:54:01,877 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-04 10:54:01,877 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:01,877 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-04 10:54:01,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:01,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:01,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:01,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:01,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:01,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:01,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:01,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:01,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:01,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,37343,1683197625438': 2023-05-04 10:54:01,880 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-04 10:54:01,880 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-04 10:54:01,880 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:54:01,880 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:54:01,881 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-04 10:54:01,881 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:54:01,938 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,938 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:01,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:01,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,938 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:01,938 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,938 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,938 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:01,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,938 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:01,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:01,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:01,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,938 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:01,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:01,939 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:01,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:01,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:01,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:01,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:01,940 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:01,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:01,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:01,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:01,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:01,942 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:01,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:01,942 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:01,943 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:01,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:01,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:01,968 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:01,968 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:01,969 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-04 10:54:01,969 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:01,969 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:54:01,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:01,969 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:01,969 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:01,968 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:01,968 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:01,969 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:01,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:01,969 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,969 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-04 10:54:01,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:01,969 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:54:01,969 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:01,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:01,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:01,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:01,969 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:01,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:01,969 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:01,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:01,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:01,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:01,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:01,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:01,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:01,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:01,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:01,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:01,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:02,069 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:54:02,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:54:02,087 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:54:02,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-04 10:54:02,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-04 10:54:02,089 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-04 10:54:02,089 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:02,089 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-04 10:54:02,089 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-04 10:54:02,090 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,090 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:02,115 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,115 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:02,115 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:02,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:02,115 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:02,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:02,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:02,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:02,115 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:02,115 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:54:02,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:02,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:02,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:02,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:02,116 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,116 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,116 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:02,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:02,116 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-04 10:54:02,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:02,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:02,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:02,116 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-04 10:54:02,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:02,116 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-04 10:54:02,117 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-04 10:54:02,117 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-04 10:54:02,117 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-04 10:54:02,117 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-04 10:54:02,117 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:02,117 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-04 10:54:02,117 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-04 10:54:02,117 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-04 10:54:02,117 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-04 10:54:02,117 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:54:02,117 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:02,117 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-04 10:54:02,117 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:02,117 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:54:02,117 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-04 10:54:02,117 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-04 10:54:02,117 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:54:02,117 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-04 10:54:02,117 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-04 10:54:02,117 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:54:02,118 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:54:02,118 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:02,118 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. started... 2023-05-04 10:54:02,118 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ff484ddaba859c44b9223e0332036369 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:54:02,129 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf1/4698bb02433b49e1ae95375348a266d8 2023-05-04 10:54:02,130 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,130 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,130 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,130 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,130 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:02,130 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:02,144 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf2/6e57600a9bb34cb58b0621c5c4694a23 2023-05-04 10:54:02,149 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf1/4698bb02433b49e1ae95375348a266d8 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/4698bb02433b49e1ae95375348a266d8 2023-05-04 10:54:02,154 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/4698bb02433b49e1ae95375348a266d8, entries=10, sequenceid=32, filesize=5.0 K 2023-05-04 10:54:02,155 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf2/6e57600a9bb34cb58b0621c5c4694a23 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6e57600a9bb34cb58b0621c5c4694a23 2023-05-04 10:54:02,159 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6e57600a9bb34cb58b0621c5c4694a23, entries=10, sequenceid=32, filesize=5.0 K 2023-05-04 10:54:02,160 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ff484ddaba859c44b9223e0332036369 in 42ms, sequenceid=32, compaction requested=true 2023-05-04 10:54:02,160 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ff484ddaba859c44b9223e0332036369: 2023-05-04 10:54:02,161 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:02,161 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:54:02,161 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:54:02,161 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,161 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-04 10:54:02,161 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-04 10:54:02,238 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,238 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:02,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:02,238 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,238 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:02,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:02,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:02,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:02,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:02,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:02,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:02,240 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-04 10:54:02,240 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6fc1bc16[Count = 0] remaining members to acquire global barrier 2023-05-04 10:54:02,240 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-04 10:54:02,240 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,284 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,284 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,284 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,284 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,284 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:54:02,284 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-04 10:54:02,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,284 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:54:02,284 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:54:02,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,285 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:54:02,285 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-04 10:54:02,285 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:54:02,285 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:54:02,285 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-04 10:54:02,285 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:54:02,330 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-04 10:54:02,330 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,330 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:02,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:02,330 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-04 10:54:02,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:02,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:02,361 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-04 10:54:02,361 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-04 10:54:02,361 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:02,361 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:02,361 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-04 10:54:02,361 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-04 10:54:02,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:02,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:02,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:02,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:02,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:02,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:02,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:02,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:02,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,37343,1683197625438': 2023-05-04 10:54:02,364 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-04 10:54:02,364 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-04 10:54:02,364 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:54:02,364 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:54:02,364 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-04 10:54:02,364 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:54:02,384 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,384 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:02,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:02,384 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,384 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,384 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:02,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:02,384 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:02,384 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:02,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,384 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:02,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:02,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:02,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:02,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:02,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:02,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:02,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:02,385 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,386 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:02,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:02,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:02,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:02,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:02,388 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:02,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:02,388 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:02,389 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:02,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:02,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:02,517 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:02,517 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:02,517 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:02,517 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:02,517 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:02,517 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:02,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:02,517 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:02,517 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-04 10:54:02,518 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:54:02,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:02,517 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:02,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:02,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:02,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:02,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:02,518 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-04 10:54:02,518 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:54:02,517 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:02,518 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:02,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:02,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:02,518 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:02,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:02,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:02,518 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:02,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:02,519 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:02,519 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,519 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,519 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:02,519 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,519 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,618 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:54:02,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:54:02,635 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:54:02,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-04 10:54:02,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-04 10:54:02,637 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-04 10:54:02,637 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:02,637 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-04 10:54:02,637 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-04 10:54:02,637 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,638 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:02,661 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:02,661 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:02,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:02,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:02,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:02,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:02,661 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:02,661 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:02,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:02,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:02,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:02,662 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:02,662 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:54:02,662 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,662 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,662 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:02,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:02,662 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-04 10:54:02,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:02,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:02,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:02,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:02,662 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-04 10:54:02,663 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-04 10:54:02,663 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-04 10:54:02,663 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-04 10:54:02,663 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-04 10:54:02,663 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:02,663 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-04 10:54:02,663 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:54:02,663 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-04 10:54:02,663 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-04 10:54:02,663 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-04 10:54:02,663 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-04 10:54:02,663 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-04 10:54:02,663 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-04 10:54:02,663 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:02,663 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-04 10:54:02,663 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:54:02,663 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:02,663 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:54:02,663 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-04 10:54:02,663 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-04 10:54:02,663 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:54:02,664 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:54:02,664 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:02,664 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. started... 2023-05-04 10:54:02,664 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ff484ddaba859c44b9223e0332036369 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:54:02,674 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf1/ee3b4e8694dc4e769030de02c20e049c 2023-05-04 10:54:02,687 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf2/6edd4c11b0904116a4c5d910b1b0aed0 2023-05-04 10:54:02,692 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf1/ee3b4e8694dc4e769030de02c20e049c as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/ee3b4e8694dc4e769030de02c20e049c 2023-05-04 10:54:02,696 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/ee3b4e8694dc4e769030de02c20e049c, entries=10, sequenceid=45, filesize=5.0 K 2023-05-04 10:54:02,697 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf2/6edd4c11b0904116a4c5d910b1b0aed0 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6edd4c11b0904116a4c5d910b1b0aed0 2023-05-04 10:54:02,702 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6edd4c11b0904116a4c5d910b1b0aed0, entries=10, sequenceid=45, filesize=5.0 K 2023-05-04 10:54:02,703 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ff484ddaba859c44b9223e0332036369 in 39ms, sequenceid=45, compaction requested=true 2023-05-04 10:54:02,703 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ff484ddaba859c44b9223e0332036369: 2023-05-04 10:54:02,703 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:02,703 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:54:02,703 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:54:02,704 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,704 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-04 10:54:02,704 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-04 10:54:02,722 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,761 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,761 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,761 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:02,782 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-04 10:54:02,815 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,815 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,815 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,815 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:02,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:02,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:02,815 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,815 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:02,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:02,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:02,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:02,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:02,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:02,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:02,817 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-04 10:54:02,817 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56cfbd4b[Count = 0] remaining members to acquire global barrier 2023-05-04 10:54:02,817 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-04 10:54:02,817 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,876 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,876 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,876 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,876 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:54:02,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,876 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-04 10:54:02,876 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,876 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:54:02,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,876 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:54:02,876 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:54:02,876 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-04 10:54:02,877 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:54:02,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,877 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:54:02,877 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-04 10:54:02,877 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:54:02,884 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-04 10:54:02,884 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-04 10:54:02,884 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,884 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:02,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:02,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:02,884 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-04 10:54:02,884 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:02,884 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-04 10:54:02,884 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:02,884 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-04 10:54:02,884 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-04 10:54:02,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:02,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:02,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:02,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:02,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:02,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:02,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:02,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:02,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:02,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,37343,1683197625438': 2023-05-04 10:54:02,888 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-04 10:54:02,888 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-04 10:54:02,888 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:54:02,888 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:54:02,888 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-04 10:54:02,888 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:54:02,915 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,915 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,915 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,915 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,915 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:02,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,915 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,915 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:02,915 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:02,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:02,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:02,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:02,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:02,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:02,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:02,916 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:02,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:02,916 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:02,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:02,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:02,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:02,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,918 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:02,918 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:02,918 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:02,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:02,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:02,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:02,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:02,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:02,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:02,970 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-04 10:54:02,970 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:54:02,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:02,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:02,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:02,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:02,970 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:02,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:02,971 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-04 10:54:02,971 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:54:02,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:02,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:02,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:02,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:02,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,970 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:02,971 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:02,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:02,971 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:02,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:02,971 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:02,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:02,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:02,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:02,971 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:02,972 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:02,972 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:02,972 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:02,972 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,071 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:54:03,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:54:03,086 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:54:03,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-04 10:54:03,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-04 10:54:03,088 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-04 10:54:03,088 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:03,088 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-04 10:54:03,088 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-04 10:54:03,089 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,089 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:03,122 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,122 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:03,122 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:03,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:03,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:03,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:03,122 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:03,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:03,123 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,123 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:54:03,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:03,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:03,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:03,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:03,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:03,123 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,123 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,123 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:03,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:03,124 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-04 10:54:03,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:03,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:03,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:03,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:03,124 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-04 10:54:03,124 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-04 10:54:03,124 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-04 10:54:03,124 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-04 10:54:03,124 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-04 10:54:03,124 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-04 10:54:03,124 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-04 10:54:03,124 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-04 10:54:03,124 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-04 10:54:03,124 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-04 10:54:03,125 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-04 10:54:03,125 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:03,125 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:03,125 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:54:03,125 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:54:03,125 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-04 10:54:03,124 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:03,125 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-04 10:54:03,125 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-04 10:54:03,125 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:54:03,125 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-04 10:54:03,125 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:54:03,125 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:54:03,125 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:03,125 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. started... 2023-05-04 10:54:03,125 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ff484ddaba859c44b9223e0332036369 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:54:03,143 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf1/92e8b67b75464deda18867aefe80112e 2023-05-04 10:54:03,155 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf2/39f13a10a1b442b9a1a083147511b745 2023-05-04 10:54:03,160 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf1/92e8b67b75464deda18867aefe80112e as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/92e8b67b75464deda18867aefe80112e 2023-05-04 10:54:03,164 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/92e8b67b75464deda18867aefe80112e, entries=10, sequenceid=58, filesize=5.0 K 2023-05-04 10:54:03,165 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf2/39f13a10a1b442b9a1a083147511b745 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/39f13a10a1b442b9a1a083147511b745 2023-05-04 10:54:03,169 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/39f13a10a1b442b9a1a083147511b745, entries=10, sequenceid=58, filesize=5.0 K 2023-05-04 10:54:03,170 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ff484ddaba859c44b9223e0332036369 in 44ms, sequenceid=58, compaction requested=true 2023-05-04 10:54:03,170 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ff484ddaba859c44b9223e0332036369: 2023-05-04 10:54:03,170 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:03,170 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:54:03,170 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:54:03,170 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,170 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-04 10:54:03,170 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-04 10:54:03,199 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:03,199 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:03,299 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:03,299 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:03,299 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:03,299 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:03,299 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,299 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:03,300 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:03,300 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:03,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:03,302 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-04 10:54:03,302 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@f48c67d[Count = 0] remaining members to acquire global barrier 2023-05-04 10:54:03,302 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-04 10:54:03,302 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:03,369 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:03,369 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:03,369 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:03,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:03,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:03,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:03,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:03,369 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:54:03,369 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-04 10:54:03,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:03,369 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:54:03,369 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:54:03,369 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-04 10:54:03,369 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:54:03,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:03,369 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:54:03,369 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-04 10:54:03,369 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:54:03,370 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,370 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:54:03,615 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-04 10:54:03,615 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-04 10:54:03,615 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:03,615 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:03,615 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-04 10:54:03,615 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-04 10:54:03,661 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,661 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-04 10:54:03,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,661 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:03,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:03,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:03,661 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-04 10:54:03,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:03,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:03,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:03,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:03,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:03,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:03,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:03,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:03,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:03,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,37343,1683197625438': 2023-05-04 10:54:03,664 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-04 10:54:03,664 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-04 10:54:03,664 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:54:03,665 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:54:03,665 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-04 10:54:03,665 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:54:03,715 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,715 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,715 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,715 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,715 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:03,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,715 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:03,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,715 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:03,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,715 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:03,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:03,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:03,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:03,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:03,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:03,716 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:03,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:03,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:03,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:03,716 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:03,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:03,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:03,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:03,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:03,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,718 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:03,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:03,718 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:03,719 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:03,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:03,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:03,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:03,761 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:03,761 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:03,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:03,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:03,761 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:03,761 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:03,761 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:03,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:03,761 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:03,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:03,761 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:03,761 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-04 10:54:03,761 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:54:03,761 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:03,761 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:03,761 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:03,762 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:03,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:03,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:03,762 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:03,762 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-04 10:54:03,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:03,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:03,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:03,762 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:54:03,762 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:03,762 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:03,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:03,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:03,762 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,762 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:03,762 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:03,762 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,862 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:54:03,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:54:03,880 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:54:03,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-04 10:54:03,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-04 10:54:03,882 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-04 10:54:03,882 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:03,882 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-04 10:54:03,882 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-04 10:54:03,882 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,882 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:03,969 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:03,969 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:03,969 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:03,969 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:03,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:03,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:03,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:03,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:03,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:03,969 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:03,969 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:54:03,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:03,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:03,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:03,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:03,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:03,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:03,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:03,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:03,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:03,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:03,970 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-04 10:54:03,970 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-04 10:54:03,970 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-04 10:54:03,971 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-04 10:54:03,971 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-04 10:54:03,971 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-04 10:54:03,971 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-04 10:54:03,971 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-04 10:54:03,971 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:03,971 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-04 10:54:03,971 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:54:03,971 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-04 10:54:03,971 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:03,971 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-04 10:54:03,971 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:54:03,971 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-04 10:54:03,971 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-04 10:54:03,971 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:54:03,971 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:54:03,971 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-04 10:54:03,971 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:03,971 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:03,971 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. started... 2023-05-04 10:54:03,972 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:54:03,972 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-04 10:54:03,972 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-04 10:54:03,972 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ff484ddaba859c44b9223e0332036369 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:54:03,983 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf1/9c7a105fef634130a447ddae622db8bd 2023-05-04 10:54:04,001 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf2/a9409bbf90bd4ef199b28a25d5239218 2023-05-04 10:54:04,006 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf1/9c7a105fef634130a447ddae622db8bd as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/9c7a105fef634130a447ddae622db8bd 2023-05-04 10:54:04,007 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:04,011 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/9c7a105fef634130a447ddae622db8bd, entries=10, sequenceid=71, filesize=5.0 K 2023-05-04 10:54:04,012 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf2/a9409bbf90bd4ef199b28a25d5239218 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/a9409bbf90bd4ef199b28a25d5239218 2023-05-04 10:54:04,019 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/a9409bbf90bd4ef199b28a25d5239218, entries=10, sequenceid=71, filesize=5.0 K 2023-05-04 10:54:04,020 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ff484ddaba859c44b9223e0332036369 in 48ms, sequenceid=71, compaction requested=true 2023-05-04 10:54:04,020 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ff484ddaba859c44b9223e0332036369: 2023-05-04 10:54:04,020 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:54:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:54:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-04 10:54:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-04 10:54:04,076 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:04,076 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:04,076 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:04,122 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:04,122 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:04,122 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:04,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:04,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:04,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:04,123 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:04,123 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:04,123 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:04,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:04,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:04,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:04,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:04,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:04,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:04,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:04,125 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-04 10:54:04,125 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45db4ac8[Count = 0] remaining members to acquire global barrier 2023-05-04 10:54:04,125 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-04 10:54:04,126 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:04,207 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:04,207 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:04,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:04,207 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:04,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:04,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:04,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:04,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:04,207 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:54:04,207 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:54:04,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:04,208 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-04 10:54:04,207 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-04 10:54:04,208 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:54:04,208 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:54:04,208 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-04 10:54:04,208 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:54:04,208 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:54:04,208 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:04,208 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:54:04,253 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-04 10:54:04,253 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-04 10:54:04,253 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:04,253 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:04,253 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-04 10:54:04,253 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-04 10:54:04,315 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-04 10:54:04,315 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:04,316 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:04,316 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-04 10:54:04,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:04,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:04,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:04,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:04,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:04,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:04,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:04,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:04,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:04,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:04,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:04,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:04,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:04,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:04,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,37343,1683197625438': 2023-05-04 10:54:04,320 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-04 10:54:04,320 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-04 10:54:04,320 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:54:04,320 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:54:04,320 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-04 10:54:04,320 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:54:04,361 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:04,361 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:04,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:04,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:04,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:04,361 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:04,361 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:04,361 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:04,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:04,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:04,361 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:04,361 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:04,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:04,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:04,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:04,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:04,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:04,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:04,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:04,361 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:04,362 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:04,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:04,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:04,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:04,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:04,362 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:04,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:04,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:04,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:04,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:04,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:04,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:04,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:04,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:04,364 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:04,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:04,364 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:04,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:04,365 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:04,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-04 10:54:04,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:04,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:04,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:04,472 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:04,472 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:04,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:04,472 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:04,472 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-04 10:54:04,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:04,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:04,472 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:04,472 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:54:04,472 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:04,472 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:04,472 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:04,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:04,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:04,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:04,472 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:04,472 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-04 10:54:04,472 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:04,472 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:54:04,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:04,472 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:04,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:04,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:04,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:04,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:04,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:04,473 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:04,473 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-04 10:54:04,473 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:04,473 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:04,473 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:04,473 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:04,473 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-04 10:54:04,473 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-04 10:54:04,573 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:54:04,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:54:04,833 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-05-04 10:54:04,834 DEBUG [Listener at localhost/35483] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:04,835 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:04,837 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:04,837 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:04,837 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:04,838 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:54:04,838 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:04,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:54:04,838 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:54:05,086 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,086 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:54:05,086 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:54:05,086 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:54:05,086 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-05-04 10:54:05,086 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:54:05,086 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:54:05,086 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:54:05,087 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:54:05,087 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:54:05,089 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-05-04 10:54:05,089 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:54:05,089 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:54:05,089 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:54:05,090 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:54:05,090 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:05,090 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:54:05,090 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:05,090 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:05,091 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:05,091 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:05,091 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:05,091 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:05,091 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:05,092 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,092 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:05,092 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,092 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,092 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,092 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-04 10:54:05,093 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:54:05,093 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:54:05,094 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197675094 2023-05-04 10:54:05,095 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:54:05,095 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,098 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:05,098 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ff484ddaba859c44b9223e0332036369, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:05,098 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:54:05,098 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197705098 2023-05-04 10:54:05,098 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197765098 2023-05-04 10:54:05,098 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:54:05,098 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:54:05,100 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:54:05,100 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:54:05,100 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:54:05,348 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,349 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:54:05,349 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:54:05,349 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:54:05,350 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:54:05,350 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:54:05,350 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:54:05,350 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:54:05,350 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:54:05,351 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:54:05,352 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-05-04 10:54:05,353 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:54:05,353 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:54:05,353 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:54:05,353 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:54:05,354 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:05,354 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:54:05,354 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:05,354 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:05,354 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:05,355 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:05,355 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:05,355 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:05,355 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:05,356 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,356 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:05,356 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,356 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,356 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,356 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-04 10:54:05,357 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:54:05,357 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:54:05,358 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197675358 2023-05-04 10:54:05,360 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,360 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:54:05,363 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:05,363 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ff484ddaba859c44b9223e0332036369, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:05,363 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:54:05,363 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197705363 2023-05-04 10:54:05,363 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197765363 2023-05-04 10:54:05,363 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:54:05,363 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:54:05,365 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN}] 2023-05-04 10:54:05,366 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:54:05,366 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,366 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:54:05,366 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:54:05,613 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,614 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:54:05,614 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:54:05,614 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:54:05,616 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:54:05,616 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:54:05,616 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:54:05,616 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:54:05,616 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:54:05,617 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:54:05,618 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-05-04 10:54:05,618 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:54:05,619 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:54:05,619 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:54:05,619 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:54:05,619 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:05,620 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:54:05,620 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:05,620 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:05,620 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:05,621 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:05,621 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:05,621 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:05,622 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:05,622 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,622 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:05,622 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,622 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,622 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,622 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-04 10:54:05,623 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:54:05,623 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:54:05,624 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197675624 2023-05-04 10:54:05,625 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,625 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:54:05,630 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:05,630 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ff484ddaba859c44b9223e0332036369, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:05,630 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:54:05,630 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197705630 2023-05-04 10:54:05,630 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197765630 2023-05-04 10:54:05,631 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-05-04 10:54:05,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN}] 2023-05-04 10:54:05,631 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:54:05,633 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:05,634 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=ff484ddaba859c44b9223e0332036369, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:05,634 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197645634"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197645634"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197645634"}]},"ts":"1683197645634"} 2023-05-04 10:54:05,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438}] 2023-05-04 10:54:05,635 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:54:05,635 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:05,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:54:05,635 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:54:05,881 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,881 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:54:05,881 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:54:05,881 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:54:05,881 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:54:05,881 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:54:05,881 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:54:05,881 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:54:05,882 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:54:05,882 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:54:05,883 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-04 10:54:05,883 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-04 10:54:05,884 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:54:05,884 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:54:05,884 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:54:05,884 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:54:05,885 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:05,885 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:54:05,885 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:05,885 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:05,885 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:05,886 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:05,886 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:05,886 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:05,886 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:05,887 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,887 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:05,887 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:05,887 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,887 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,887 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:05,887 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:05,887 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:05,888 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-04 10:54:05,888 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:54:05,889 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:54:05,889 INFO [Listener at localhost/35483] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369 to restore RIT 2023-05-04 10:54:05,889 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197675889 2023-05-04 10:54:05,890 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:05,890 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:54:05,893 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:05,893 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ff484ddaba859c44b9223e0332036369, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:05,893 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:54:05,894 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197705894 2023-05-04 10:54:05,894 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197765894 2023-05-04 10:54:05,894 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:54:05,894 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=ff484ddaba859c44b9223e0332036369, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:05,894 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197645894"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197645894"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197645894"}]},"ts":"1683197645894"} 2023-05-04 10:54:05,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438}] 2023-05-04 10:54:05,896 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:54:06,049 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,049 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ff484ddaba859c44b9223e0332036369, disabling compactions & flushes 2023-05-04 10:54:06,049 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:06,049 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:06,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. after waiting 0 ms 2023-05-04 10:54:06,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:06,050 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ff484ddaba859c44b9223e0332036369 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-04 10:54:06,060 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf1/f43c80d29349442f9ec4e43f526ce01c 2023-05-04 10:54:06,074 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf2/98c7029ed47a4654bd13dbafdeb4f68d 2023-05-04 10:54:06,079 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf1/f43c80d29349442f9ec4e43f526ce01c as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/f43c80d29349442f9ec4e43f526ce01c 2023-05-04 10:54:06,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/f43c80d29349442f9ec4e43f526ce01c, entries=8, sequenceid=82, filesize=5.0 K 2023-05-04 10:54:06,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf2/98c7029ed47a4654bd13dbafdeb4f68d as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/98c7029ed47a4654bd13dbafdeb4f68d 2023-05-04 10:54:06,090 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/98c7029ed47a4654bd13dbafdeb4f68d, entries=8, sequenceid=82, filesize=5.0 K 2023-05-04 10:54:06,091 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ff484ddaba859c44b9223e0332036369 in 41ms, sequenceid=82, compaction requested=true 2023-05-04 10:54:06,096 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-04 10:54:06,096 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:54:06,097 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:06,097 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ff484ddaba859c44b9223e0332036369: 2023-05-04 10:54:06,100 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,100 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=ff484ddaba859c44b9223e0332036369, regionState=CLOSED 2023-05-04 10:54:06,100 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197646100"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197646100"}]},"ts":"1683197646100"} 2023-05-04 10:54:06,102 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:54:06,102 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:06,102 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:54:06,102 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:54:06,144 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:06,144 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:54:06,144 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:54:06,144 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:54:06,145 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:54:06,145 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:54:06,145 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:54:06,146 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:54:06,146 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:54:06,146 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:54:06,147 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-04 10:54:06,147 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-05-04 10:54:06,147 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-05-04 10:54:06,148 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:54:06,148 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:54:06,148 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:54:06,148 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:54:06,149 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:06,149 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:54:06,149 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:06,149 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:06,149 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:06,150 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:06,150 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:06,150 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:06,150 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:06,151 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:06,151 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:06,151 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:06,151 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:06,151 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:06,151 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:06,151 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:06,151 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:06,151 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:06,151 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:06,152 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-04 10:54:06,152 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:54:06,153 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:54:06,153 INFO [Listener at localhost/35483] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369 to restore RIT 2023-05-04 10:54:06,154 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197676154 2023-05-04 10:54:06,155 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:06,155 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:54:06,162 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:06,163 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ff484ddaba859c44b9223e0332036369, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:06,163 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:54:06,163 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197706163 2023-05-04 10:54:06,163 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197766163 2023-05-04 10:54:06,163 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 8 msec 2023-05-04 10:54:06,164 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=ff484ddaba859c44b9223e0332036369, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:06,164 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197646163"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197646163"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197646163"}]},"ts":"1683197646163"} 2023-05-04 10:54:06,165 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:54:06,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-05-04 10:54:06,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 in 269 msec 2023-05-04 10:54:06,169 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:54:06,169 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:06,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:54:06,169 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:54:06,414 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:06,414 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:54:06,414 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:54:06,414 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:54:06,414 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:54:06,414 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:54:06,415 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:54:06,415 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:54:06,415 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:54:06,416 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:54:06,417 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-04 10:54:06,417 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-05-04 10:54:06,417 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:54:06,418 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:54:06,418 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:54:06,419 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:54:06,419 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:54:06,419 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:06,420 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:54:06,420 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:06,420 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:06,420 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:06,421 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:06,421 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:06,421 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:06,422 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:06,423 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:06,423 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:06,423 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:06,423 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:06,423 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:06,423 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:06,424 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:06,424 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:06,424 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:06,424 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-05-04 10:54:06,425 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:54:06,425 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:54:06,425 INFO [Listener at localhost/35483] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369 to restore RIT 2023-05-04 10:54:06,426 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197676426 2023-05-04 10:54:06,427 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:06,427 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:54:06,429 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:06,430 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ff484ddaba859c44b9223e0332036369, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:06,430 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:54:06,430 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197706430 2023-05-04 10:54:06,430 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197766430 2023-05-04 10:54:06,430 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:54:06,430 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:06,432 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:54:06,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-05-04 10:54:06,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN in 801 msec 2023-05-04 10:54:06,438 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:54:06,438 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:06,438 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:54:06,438 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:54:06,681 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:06,681 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:54:06,681 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:54:06,681 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:54:06,682 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:54:06,682 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:54:06,682 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:54:06,682 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:54:06,683 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:54:06,684 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:54:06,685 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-04 10:54:06,685 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:54:06,685 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:54:06,685 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:54:06,686 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:54:06,686 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:54:06,686 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:54:06,687 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:06,687 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:54:06,687 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:06,688 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:06,688 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:06,688 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:06,689 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:06,689 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:06,689 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:06,690 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:06,690 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:06,690 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:06,690 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:06,690 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:06,690 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:06,691 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-04 10:54:06,691 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:54:06,692 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:54:06,693 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197676693 2023-05-04 10:54:06,695 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:06,695 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:54:06,698 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:06,698 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ff484ddaba859c44b9223e0332036369, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:06,698 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:54:06,698 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197706698 2023-05-04 10:54:06,698 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197766698 2023-05-04 10:54:06,698 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:54:06,699 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:06,699 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:06,699 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:06,703 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:54:06,712 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=ff484ddaba859c44b9223e0332036369, threads=14 2023-05-04 10:54:06,713 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/39f13a10a1b442b9a1a083147511b745 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,713 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6e57600a9bb34cb58b0621c5c4694a23 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,713 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6edd4c11b0904116a4c5d910b1b0aed0 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,713 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/75b40abe85aa4e2fafa1aac676320bf1 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,714 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/98c7029ed47a4654bd13dbafdeb4f68d for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,714 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/a9409bbf90bd4ef199b28a25d5239218 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,714 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/ea0f76018bc64a8383fdbba25fa2a396 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,714 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/4698bb02433b49e1ae95375348a266d8 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,717 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/81ce75c883534f40a8d1da89d3df7e7c for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,717 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/92e8b67b75464deda18867aefe80112e for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,717 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/9c7a105fef634130a447ddae622db8bd for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,718 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/de0e333521134ffb8370a273ad000f44 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,718 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/ee3b4e8694dc4e769030de02c20e049c for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,718 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/f43c80d29349442f9ec4e43f526ce01c for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,907 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/39f13a10a1b442b9a1a083147511b745 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,941 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/9c7a105fef634130a447ddae622db8bd for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,942 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/a9409bbf90bd4ef199b28a25d5239218 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,944 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6edd4c11b0904116a4c5d910b1b0aed0 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,951 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/98c7029ed47a4654bd13dbafdeb4f68d for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,960 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/f43c80d29349442f9ec4e43f526ce01c for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,962 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/81ce75c883534f40a8d1da89d3df7e7c for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,962 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/4698bb02433b49e1ae95375348a266d8 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,963 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/ea0f76018bc64a8383fdbba25fa2a396 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,968 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/ee3b4e8694dc4e769030de02c20e049c for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,968 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6e57600a9bb34cb58b0621c5c4694a23 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,976 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/de0e333521134ffb8370a273ad000f44 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,982 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/92e8b67b75464deda18867aefe80112e for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,983 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/75b40abe85aa4e2fafa1aac676320bf1 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:06,983 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region ff484ddaba859c44b9223e0332036369 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-04 10:54:07,024 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:54:07,025 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:54:07,025 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:54:07,199 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,199 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:54:07,199 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:54:07,199 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:54:07,199 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:54:07,199 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:54:07,199 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:54:07,199 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:54:07,200 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:54:07,200 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:54:07,201 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-05-04 10:54:07,201 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:54:07,202 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:54:07,202 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:54:07,202 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:54:07,203 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:54:07,203 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:54:07,203 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:07,203 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:54:07,203 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:07,204 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:07,204 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:07,204 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:07,204 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:07,204 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:07,205 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:07,205 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,205 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:07,205 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:07,205 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:07,205 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,205 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,206 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,206 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:07,206 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:07,206 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-04 10:54:07,206 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:54:07,207 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:54:07,207 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197677207 2023-05-04 10:54:07,209 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:54:07,209 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,211 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:07,211 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ff484ddaba859c44b9223e0332036369, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:07,211 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:54:07,212 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197707212 2023-05-04 10:54:07,212 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197767212 2023-05-04 10:54:07,212 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:54:07,221 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=ff484ddaba859c44b9223e0332036369, threads=14 2023-05-04 10:54:07,222 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/39f13a10a1b442b9a1a083147511b745 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,222 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6e57600a9bb34cb58b0621c5c4694a23 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,222 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6edd4c11b0904116a4c5d910b1b0aed0 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,222 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/98c7029ed47a4654bd13dbafdeb4f68d for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,222 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/75b40abe85aa4e2fafa1aac676320bf1 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,223 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/a9409bbf90bd4ef199b28a25d5239218 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,223 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/ea0f76018bc64a8383fdbba25fa2a396 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,224 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/4698bb02433b49e1ae95375348a266d8 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,224 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/81ce75c883534f40a8d1da89d3df7e7c for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,226 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/92e8b67b75464deda18867aefe80112e for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,226 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/9c7a105fef634130a447ddae622db8bd for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,227 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/de0e333521134ffb8370a273ad000f44 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,228 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/ee3b4e8694dc4e769030de02c20e049c for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,228 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/f43c80d29349442f9ec4e43f526ce01c for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,374 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6edd4c11b0904116a4c5d910b1b0aed0 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,392 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/98c7029ed47a4654bd13dbafdeb4f68d for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,432 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/f43c80d29349442f9ec4e43f526ce01c for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,443 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/9c7a105fef634130a447ddae622db8bd for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,443 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/a9409bbf90bd4ef199b28a25d5239218 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,443 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/39f13a10a1b442b9a1a083147511b745 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,452 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/4698bb02433b49e1ae95375348a266d8 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,472 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/81ce75c883534f40a8d1da89d3df7e7c for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,487 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/ee3b4e8694dc4e769030de02c20e049c for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,488 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/ea0f76018bc64a8383fdbba25fa2a396 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,490 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/de0e333521134ffb8370a273ad000f44 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,490 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6e57600a9bb34cb58b0621c5c4694a23 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,494 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/92e8b67b75464deda18867aefe80112e for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,498 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/75b40abe85aa4e2fafa1aac676320bf1 for region: ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:07,498 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region ff484ddaba859c44b9223e0332036369 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-04 10:54:07,546 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:54:07,552 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/e232f81670423fc5259ac5116e147909/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-04 10:54:07,554 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/40856f69942775d59f85b65348effee4/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-04 10:54:07,554 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:54:07,554 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,554 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:54:07,555 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:54:07,712 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,712 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:54:07,712 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:54:07,713 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:54:07,713 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:54:07,713 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:54:07,713 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:54:07,713 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:54:07,713 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:54:07,714 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:54:07,715 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-05-04 10:54:07,716 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:54:07,716 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:54:07,716 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:54:07,717 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:54:07,717 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:54:07,717 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:54:07,717 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:07,718 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:54:07,718 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:07,718 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:07,719 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:07,719 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:07,719 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:07,719 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:07,720 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:07,720 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,720 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:07,721 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:07,723 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:07,723 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,723 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,723 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,723 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:07,723 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:07,724 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-05-04 10:54:07,724 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:54:07,725 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:54:07,725 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197677725 2023-05-04 10:54:07,726 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,726 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:54:07,729 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:07,730 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ff484ddaba859c44b9223e0332036369, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:07,730 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:54:07,730 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197707730 2023-05-04 10:54:07,730 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197767730 2023-05-04 10:54:07,730 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:54:07,733 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:54:07,736 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:54:07,736 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,736 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:54:07,736 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:54:07,981 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,981 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:54:07,981 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:54:07,981 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:54:07,983 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:54:07,983 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:54:07,983 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:54:07,983 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:54:07,983 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:54:07,984 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:54:07,985 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-05-04 10:54:07,985 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:54:07,985 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:54:07,986 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:54:07,986 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:54:07,986 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:54:07,987 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:54:07,987 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:07,987 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:54:07,988 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:07,988 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:07,988 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:07,989 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:07,989 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:07,989 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:07,989 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:07,990 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,990 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:07,990 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:07,990 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:07,991 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,991 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,991 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,991 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:07,991 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:07,991 DEBUG [Listener at localhost/35483] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,991 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-04 10:54:07,992 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:54:07,993 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:54:07,993 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197677993 2023-05-04 10:54:07,995 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:07,995 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:54:07,998 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:07,998 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ff484ddaba859c44b9223e0332036369, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:07,998 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:54:07,998 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197707998 2023-05-04 10:54:07,998 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197767998 2023-05-04 10:54:07,998 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-04 10:54:07,999 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:54:08,011 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:54:08,011 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-04 10:54:08,011 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:54:08,011 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:54:08,250 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:08,250 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:54:08,250 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:54:08,250 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:54:08,251 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:54:08,251 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:54:08,251 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:54:08,251 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:54:08,251 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:54:08,252 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:54:08,253 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-05-04 10:54:08,253 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:54:08,254 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:54:08,254 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:54:08,254 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:54:08,254 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:54:08,254 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:54:08,255 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:08,255 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:54:08,255 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:08,255 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:08,255 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:08,256 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:08,256 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:08,256 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:08,256 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:08,257 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:08,257 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:08,257 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:08,257 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:08,257 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:08,257 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:08,257 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:08,257 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:08,257 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:08,258 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-04 10:54:08,258 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:54:08,259 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:54:08,259 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197678259 2023-05-04 10:54:08,262 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:08,262 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:54:08,264 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:08,264 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ff484ddaba859c44b9223e0332036369, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:08,264 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:54:08,264 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197708264 2023-05-04 10:54:08,264 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197768264 2023-05-04 10:54:08,264 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-04 10:54:08,264 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:54:08,267 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:54:08,267 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-04 10:54:08,267 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:54:08,267 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:54:08,515 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:08,516 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:54:08,516 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:54:08,516 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:54:08,516 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:54:08,516 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:54:08,516 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:54:08,516 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:54:08,516 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:54:08,517 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:54:08,518 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-05-04 10:54:08,518 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:54:08,518 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:54:08,518 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:54:08,519 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:54:08,520 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:54:08,520 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:54:08,520 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:08,520 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:54:08,521 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:08,521 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:08,521 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:08,521 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:08,522 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:08,522 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:08,522 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:08,523 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:08,523 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:08,523 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:08,523 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:08,523 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:08,523 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:08,523 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:08,524 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:08,524 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:08,524 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-04 10:54:08,524 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:54:08,525 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:54:08,525 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197678525 2023-05-04 10:54:08,527 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:54:08,527 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:08,529 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:08,529 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ff484ddaba859c44b9223e0332036369, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:08,529 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:54:08,529 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197708529 2023-05-04 10:54:08,529 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197768529 2023-05-04 10:54:08,529 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-04 10:54:08,530 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:54:08,532 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:54:08,532 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-04 10:54:08,532 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:54:08,532 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:54:08,780 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:08,780 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:54:08,780 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:54:08,780 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:54:08,781 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:54:08,781 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:54:08,781 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:54:08,781 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:54:08,781 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:54:08,782 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:54:08,783 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-05-04 10:54:08,783 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:54:08,783 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:54:08,783 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:54:08,784 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:54:08,784 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:54:08,784 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:54:08,784 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:08,785 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:54:08,785 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:08,785 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:08,785 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:08,785 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:08,785 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:08,785 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:08,786 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:08,786 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:08,786 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:08,786 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:08,787 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:08,787 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:08,787 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:08,787 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:08,787 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:08,787 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:08,787 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-04 10:54:08,788 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:54:08,788 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:54:08,789 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197678789 2023-05-04 10:54:08,790 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:54:08,791 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:08,793 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:08,794 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ff484ddaba859c44b9223e0332036369, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:08,794 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:54:08,794 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197708794 2023-05-04 10:54:08,794 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197768794 2023-05-04 10:54:08,794 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:54:08,794 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:54:08,797 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN] 2023-05-04 10:54:08,797 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:54:08,797 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-04 10:54:08,797 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:54:08,798 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:54:09,044 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,044 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:54:09,044 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:54:09,044 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:54:09,045 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:54:09,045 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:54:09,046 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:54:09,046 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:54:09,046 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:54:09,046 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:54:09,047 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-05-04 10:54:09,047 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:54:09,047 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:54:09,047 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-04 10:54:09,047 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:54:09,048 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:54:09,048 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:54:09,048 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:54:09,048 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:09,048 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:54:09,049 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:09,049 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:09,049 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:09,049 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:09,049 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:09,049 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:09,050 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:09,051 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,051 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:09,051 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:09,051 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:54:09,051 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:09,051 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,051 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,051 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,051 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:09,051 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:09,051 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:09,051 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-04 10:54:09,052 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:54:09,053 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:54:09,053 INFO [Listener at localhost/35483] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369 to restore RIT 2023-05-04 10:54:09,053 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197679053 2023-05-04 10:54:09,054 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:54:09,055 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,056 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:54:09,058 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:09,058 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ff484ddaba859c44b9223e0332036369, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:09,058 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:54:09,058 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197709058 2023-05-04 10:54:09,058 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197769058 2023-05-04 10:54:09,058 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-04 10:54:09,059 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:54:09,063 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:54:09,063 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-04 10:54:09,063 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:54:09,063 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:54:09,309 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,309 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:54:09,309 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:54:09,309 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:54:09,311 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:54:09,311 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:54:09,311 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:54:09,311 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:54:09,312 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:54:09,312 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:54:09,313 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-04 10:54:09,313 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-04 10:54:09,313 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:54:09,314 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:54:09,314 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:54:09,314 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:54:09,314 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:09,314 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:54:09,315 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:09,315 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:09,315 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:09,315 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:09,315 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:09,315 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:09,316 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:09,316 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,316 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:54:09,316 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:09,316 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,316 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,316 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,316 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:09,317 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-04 10:54:09,317 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:54:09,318 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:54:09,318 INFO [Listener at localhost/35483] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369 to restore RIT 2023-05-04 10:54:09,318 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197679318 2023-05-04 10:54:09,319 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:54:09,320 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,321 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:54:09,323 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:09,323 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ff484ddaba859c44b9223e0332036369, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:09,323 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:54:09,323 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197709323 2023-05-04 10:54:09,323 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197769323 2023-05-04 10:54:09,323 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:54:09,323 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:54:09,325 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:54:09,325 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-04 10:54:09,325 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:54:09,325 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:54:09,573 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,573 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:54:09,573 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:54:09,574 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:54:09,574 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:54:09,574 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:54:09,574 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:54:09,574 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:54:09,574 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:54:09,575 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:54:09,576 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-05-04 10:54:09,576 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-04 10:54:09,576 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:54:09,576 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:54:09,576 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:54:09,576 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:54:09,577 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:09,577 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:54:09,577 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:09,577 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:09,578 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:09,578 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:09,578 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:09,578 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:09,578 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:09,579 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,579 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:54:09,579 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:09,579 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,579 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,579 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,579 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:09,579 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-04 10:54:09,580 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:54:09,580 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:54:09,580 INFO [Listener at localhost/35483] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369 to restore RIT 2023-05-04 10:54:09,581 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197679581 2023-05-04 10:54:09,581 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:54:09,582 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,583 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:54:09,585 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:09,585 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ff484ddaba859c44b9223e0332036369, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:09,586 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:54:09,586 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197709586 2023-05-04 10:54:09,586 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197769586 2023-05-04 10:54:09,586 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-05-04 10:54:09,586 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:54:09,588 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:54:09,588 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-04 10:54:09,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:54:09,588 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:54:09,836 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,836 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:54:09,836 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:54:09,836 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:54:09,837 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:54:09,837 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:54:09,837 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:54:09,837 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:54:09,837 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:54:09,838 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:54:09,838 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-05-04 10:54:09,839 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-04 10:54:09,839 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:54:09,839 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:54:09,839 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:54:09,839 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:54:09,840 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:09,840 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:54:09,840 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:09,840 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:09,841 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:09,841 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:09,841 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:09,841 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:09,841 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:09,842 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,842 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:54:09,842 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:09,842 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,842 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,842 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,842 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:54:09,842 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-04 10:54:09,843 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:54:09,843 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:54:09,843 INFO [Listener at localhost/35483] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369 to restore RIT 2023-05-04 10:54:09,844 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197679844 2023-05-04 10:54:09,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:09,846 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:54:09,848 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:54:09,850 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:09,850 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ff484ddaba859c44b9223e0332036369, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:09,851 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:54:09,851 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197709851 2023-05-04 10:54:09,851 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197769851 2023-05-04 10:54:09,851 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-05-04 10:54:09,851 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:54:09,853 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 exec-time=5.0170 sec 2023-05-04 10:54:09,853 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:54:09,854 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,37343,1683197625438; forceNewPlan=false, retain=true 2023-05-04 10:54:10,004 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-04 10:54:10,006 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=ff484ddaba859c44b9223e0332036369, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:10,006 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197650005"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197650005"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197650005"}]},"ts":"1683197650005"} 2023-05-04 10:54:10,008 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438}] 2023-05-04 10:54:10,008 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:54:10,008 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:54:10,008 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:54:10,008 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:54:10,101 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:54:10,101 WARN [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-05-04 10:54:10,101 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:54:10,101 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:54:10,102 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:54:10,102 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:54:10,102 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:54:10,102 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:54:10,102 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:54:10,102 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:54:10,103 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:54:10,104 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-05-04 10:54:10,105 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:54:10,105 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:54:10,105 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:54:10,105 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:54:10,106 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:10,106 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:54:10,106 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:10,107 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:54:10,107 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:10,107 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:54:10,108 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:10,108 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:54:10,108 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:10,108 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:54:10,109 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:54:10,109 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:54:10,109 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:54:10,109 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-04 10:54:10,110 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:54:10,110 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:54:10,110 INFO [Listener at localhost/35483] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369 to restore RIT 2023-05-04 10:54:10,111 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197680111 2023-05-04 10:54:10,112 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:54:10,112 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:54:10,113 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:10,114 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ff484ddaba859c44b9223e0332036369, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:54:10,114 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:54:10,114 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197710114 2023-05-04 10:54:10,114 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197770114 2023-05-04 10:54:10,114 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-04 10:54:10,114 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=ff484ddaba859c44b9223e0332036369, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:10,114 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197650114"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197650114"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197650114"}]},"ts":"1683197650114"} 2023-05-04 10:54:10,116 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438}] 2023-05-04 10:54:10,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:10,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ff484ddaba859c44b9223e0332036369, NAME => 'testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.', STARTKEY => '', ENDKEY => ''} 2023-05-04 10:54:10,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:54:10,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:10,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:54:10,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:10,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:10,273 INFO [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:10,274 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1 2023-05-04 10:54:10,274 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1 2023-05-04 10:54:10,274 INFO [StoreOpener-ff484ddaba859c44b9223e0332036369-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ff484ddaba859c44b9223e0332036369 columnFamilyName cf1 2023-05-04 10:54:10,281 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/4698bb02433b49e1ae95375348a266d8 2023-05-04 10:54:10,284 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/81ce75c883534f40a8d1da89d3df7e7c 2023-05-04 10:54:10,287 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/92e8b67b75464deda18867aefe80112e 2023-05-04 10:54:10,290 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/9c7a105fef634130a447ddae622db8bd 2023-05-04 10:54:10,293 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/de0e333521134ffb8370a273ad000f44 2023-05-04 10:54:10,295 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/ee3b4e8694dc4e769030de02c20e049c 2023-05-04 10:54:10,299 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/f43c80d29349442f9ec4e43f526ce01c 2023-05-04 10:54:10,299 INFO [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(310): Store=ff484ddaba859c44b9223e0332036369/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:54:10,299 INFO [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:10,300 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2 2023-05-04 10:54:10,300 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2 2023-05-04 10:54:10,300 INFO [StoreOpener-ff484ddaba859c44b9223e0332036369-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ff484ddaba859c44b9223e0332036369 columnFamilyName cf2 2023-05-04 10:54:10,306 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/39f13a10a1b442b9a1a083147511b745 2023-05-04 10:54:10,309 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6e57600a9bb34cb58b0621c5c4694a23 2023-05-04 10:54:10,313 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6edd4c11b0904116a4c5d910b1b0aed0 2023-05-04 10:54:10,316 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/75b40abe85aa4e2fafa1aac676320bf1 2023-05-04 10:54:10,320 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/98c7029ed47a4654bd13dbafdeb4f68d 2023-05-04 10:54:10,325 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/a9409bbf90bd4ef199b28a25d5239218 2023-05-04 10:54:10,328 DEBUG [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/ea0f76018bc64a8383fdbba25fa2a396 2023-05-04 10:54:10,328 INFO [StoreOpener-ff484ddaba859c44b9223e0332036369-1] regionserver.HStore(310): Store=ff484ddaba859c44b9223e0332036369/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:54:10,329 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:10,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:10,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.splits 2023-05-04 10:54:10,332 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:54:10,333 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:10,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ff484ddaba859c44b9223e0332036369; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74108802, jitterRate=0.10430720448493958}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:54:10,334 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ff484ddaba859c44b9223e0332036369: 2023-05-04 10:54:10,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369., pid=48, masterSystemTime=1683197650268 2023-05-04 10:54:10,335 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:54:10,335 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-04 10:54:10,336 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:54:10,336 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:54:10,337 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-05-04 10:54:10,337 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-05-04 10:54:10,337 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HStore(1912): ff484ddaba859c44b9223e0332036369/cf2 is initiating minor compaction (all files) 2023-05-04 10:54:10,338 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HStore(1912): ff484ddaba859c44b9223e0332036369/cf1 is initiating minor compaction (all files) 2023-05-04 10:54:10,338 INFO [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HRegion(2259): Starting compaction of ff484ddaba859c44b9223e0332036369/cf2 in testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:10,338 INFO [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ff484ddaba859c44b9223e0332036369/cf1 in testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:10,338 INFO [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/75b40abe85aa4e2fafa1aac676320bf1, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/ea0f76018bc64a8383fdbba25fa2a396, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6e57600a9bb34cb58b0621c5c4694a23, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6edd4c11b0904116a4c5d910b1b0aed0, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/39f13a10a1b442b9a1a083147511b745, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/a9409bbf90bd4ef199b28a25d5239218, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/98c7029ed47a4654bd13dbafdeb4f68d] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp, totalSize=34.9 K 2023-05-04 10:54:10,338 INFO [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/81ce75c883534f40a8d1da89d3df7e7c, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/de0e333521134ffb8370a273ad000f44, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/4698bb02433b49e1ae95375348a266d8, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/ee3b4e8694dc4e769030de02c20e049c, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/92e8b67b75464deda18867aefe80112e, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/9c7a105fef634130a447ddae622db8bd, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/f43c80d29349442f9ec4e43f526ce01c] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp, totalSize=34.9 K 2023-05-04 10:54:10,338 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:10,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:10,339 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting 75b40abe85aa4e2fafa1aac676320bf1, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683197641245 2023-05-04 10:54:10,339 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting 81ce75c883534f40a8d1da89d3df7e7c, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683197641245 2023-05-04 10:54:10,339 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=ff484ddaba859c44b9223e0332036369, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:10,339 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197650339"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197650339"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197650339"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197650339"}]},"ts":"1683197650339"} 2023-05-04 10:54:10,339 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting ea0f76018bc64a8383fdbba25fa2a396, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683197641448 2023-05-04 10:54:10,339 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting de0e333521134ffb8370a273ad000f44, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683197641448 2023-05-04 10:54:10,340 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting 6e57600a9bb34cb58b0621c5c4694a23, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683197642072 2023-05-04 10:54:10,340 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting 4698bb02433b49e1ae95375348a266d8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683197642072 2023-05-04 10:54:10,340 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting 6edd4c11b0904116a4c5d910b1b0aed0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683197642620 2023-05-04 10:54:10,340 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting ee3b4e8694dc4e769030de02c20e049c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683197642620 2023-05-04 10:54:10,341 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting 39f13a10a1b442b9a1a083147511b745, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683197643073 2023-05-04 10:54:10,341 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting 92e8b67b75464deda18867aefe80112e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683197643073 2023-05-04 10:54:10,341 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting a9409bbf90bd4ef199b28a25d5239218, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683197643864 2023-05-04 10:54:10,343 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-05-04 10:54:10,343 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 in 225 msec 2023-05-04 10:54:10,345 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN in 1.5470 sec 2023-05-04 10:54:10,346 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting 9c7a105fef634130a447ddae622db8bd, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683197643864 2023-05-04 10:54:10,346 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting 98c7029ed47a4654bd13dbafdeb4f68d, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683197644575 2023-05-04 10:54:10,346 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting f43c80d29349442f9ec4e43f526ce01c, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683197644575 2023-05-04 10:54:10,376 INFO [RS:0;jenkins-hbase4:37343-shortCompactions-0] throttle.PressureAwareThroughputController(145): ff484ddaba859c44b9223e0332036369#cf1#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:54:10,377 INFO [RS:0;jenkins-hbase4:37343-longCompactions-0] throttle.PressureAwareThroughputController(145): ff484ddaba859c44b9223e0332036369#cf2#compaction#51 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:54:10,404 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf2/9dbc2679fccd43f28085a92704b3ee1d as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/9dbc2679fccd43f28085a92704b3ee1d 2023-05-04 10:54:10,406 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/.tmp/cf1/75ff11b2f2754de7b85c2e83f7f8eca9 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/75ff11b2f2754de7b85c2e83f7f8eca9 2023-05-04 10:54:10,409 INFO [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ff484ddaba859c44b9223e0332036369/cf2 of ff484ddaba859c44b9223e0332036369 into 9dbc2679fccd43f28085a92704b3ee1d(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:54:10,409 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for ff484ddaba859c44b9223e0332036369: 2023-05-04 10:54:10,409 INFO [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369., storeName=ff484ddaba859c44b9223e0332036369/cf2, priority=9, startTime=1683197650335; duration=0sec 2023-05-04 10:54:10,410 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:54:10,412 INFO [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ff484ddaba859c44b9223e0332036369/cf1 of ff484ddaba859c44b9223e0332036369 into 75ff11b2f2754de7b85c2e83f7f8eca9(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:54:10,412 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ff484ddaba859c44b9223e0332036369: 2023-05-04 10:54:10,412 INFO [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369., storeName=ff484ddaba859c44b9223e0332036369/cf1, priority=9, startTime=1683197650335; duration=0sec 2023-05-04 10:54:10,412 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:54:10,691 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-05-04 10:54:10,693 DEBUG [Listener at localhost/35483] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-05-04 10:54:10,695 INFO [Listener at localhost/35483] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-05-04 10:54:10,697 INFO [Listener at localhost/35483] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-05-04 10:54:10,697 INFO [Listener at localhost/35483] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-05-04 10:54:10,698 INFO [Listener at localhost/35483] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-05-04 10:54:10,714 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:54:10,716 INFO [Listener at localhost/35483] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-05-04 10:54:10,716 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-05-04 10:54:10,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:10,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-05-04 10:54:10,720 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197650720"}]},"ts":"1683197650720"} 2023-05-04 10:54:10,721 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-05-04 10:54:10,722 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-05-04 10:54:10,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN}] 2023-05-04 10:54:10,724 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN 2023-05-04 10:54:10,724 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=ff484ddaba859c44b9223e0332036369, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:10,725 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197650724"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197650724"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197650724"}]},"ts":"1683197650724"} 2023-05-04 10:54:10,726 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438}] 2023-05-04 10:54:10,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-05-04 10:54:10,877 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:10,878 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ff484ddaba859c44b9223e0332036369, disabling compactions & flushes 2023-05-04 10:54:10,878 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:10,878 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:10,878 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. after waiting 0 ms 2023-05-04 10:54:10,878 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:10,879 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/81ce75c883534f40a8d1da89d3df7e7c, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/de0e333521134ffb8370a273ad000f44, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/4698bb02433b49e1ae95375348a266d8, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/ee3b4e8694dc4e769030de02c20e049c, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/92e8b67b75464deda18867aefe80112e, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/9c7a105fef634130a447ddae622db8bd, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/f43c80d29349442f9ec4e43f526ce01c] to archive 2023-05-04 10:54:10,880 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:54:10,882 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/81ce75c883534f40a8d1da89d3df7e7c to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/81ce75c883534f40a8d1da89d3df7e7c 2023-05-04 10:54:10,883 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/de0e333521134ffb8370a273ad000f44 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/de0e333521134ffb8370a273ad000f44 2023-05-04 10:54:10,884 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/4698bb02433b49e1ae95375348a266d8 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/4698bb02433b49e1ae95375348a266d8 2023-05-04 10:54:10,886 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/ee3b4e8694dc4e769030de02c20e049c to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/ee3b4e8694dc4e769030de02c20e049c 2023-05-04 10:54:10,887 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/92e8b67b75464deda18867aefe80112e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/92e8b67b75464deda18867aefe80112e 2023-05-04 10:54:10,889 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/9c7a105fef634130a447ddae622db8bd to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/9c7a105fef634130a447ddae622db8bd 2023-05-04 10:54:10,890 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/f43c80d29349442f9ec4e43f526ce01c to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/f43c80d29349442f9ec4e43f526ce01c 2023-05-04 10:54:10,896 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/75b40abe85aa4e2fafa1aac676320bf1, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/ea0f76018bc64a8383fdbba25fa2a396, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6e57600a9bb34cb58b0621c5c4694a23, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6edd4c11b0904116a4c5d910b1b0aed0, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/39f13a10a1b442b9a1a083147511b745, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/a9409bbf90bd4ef199b28a25d5239218, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/98c7029ed47a4654bd13dbafdeb4f68d] to archive 2023-05-04 10:54:10,897 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:54:10,898 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/75b40abe85aa4e2fafa1aac676320bf1 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/75b40abe85aa4e2fafa1aac676320bf1 2023-05-04 10:54:10,899 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/ea0f76018bc64a8383fdbba25fa2a396 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/ea0f76018bc64a8383fdbba25fa2a396 2023-05-04 10:54:10,900 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6e57600a9bb34cb58b0621c5c4694a23 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6e57600a9bb34cb58b0621c5c4694a23 2023-05-04 10:54:10,901 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6edd4c11b0904116a4c5d910b1b0aed0 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/6edd4c11b0904116a4c5d910b1b0aed0 2023-05-04 10:54:10,902 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/39f13a10a1b442b9a1a083147511b745 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/39f13a10a1b442b9a1a083147511b745 2023-05-04 10:54:10,904 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/a9409bbf90bd4ef199b28a25d5239218 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/a9409bbf90bd4ef199b28a25d5239218 2023-05-04 10:54:10,905 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/98c7029ed47a4654bd13dbafdeb4f68d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/98c7029ed47a4654bd13dbafdeb4f68d 2023-05-04 10:54:10,909 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-04 10:54:10,909 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:54:10,910 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369. 2023-05-04 10:54:10,910 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ff484ddaba859c44b9223e0332036369: 2023-05-04 10:54:10,911 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:10,912 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=ff484ddaba859c44b9223e0332036369, regionState=CLOSED 2023-05-04 10:54:10,912 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197650912"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197650912"}]},"ts":"1683197650912"} 2023-05-04 10:54:10,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-05-04 10:54:10,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure ff484ddaba859c44b9223e0332036369, server=jenkins-hbase4.apache.org,37343,1683197625438 in 187 msec 2023-05-04 10:54:10,916 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-05-04 10:54:10,917 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, UNASSIGN in 192 msec 2023-05-04 10:54:10,917 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197650917"}]},"ts":"1683197650917"} 2023-05-04 10:54:10,918 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-05-04 10:54:10,921 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-05-04 10:54:10,923 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 206 msec 2023-05-04 10:54:11,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-05-04 10:54:11,022 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-05-04 10:54:11,023 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-05-04 10:54:11,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:11,026 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:11,026 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:11,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-05-04 10:54:11,030 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:11,033 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/recovered.edits] 2023-05-04 10:54:11,035 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/75ff11b2f2754de7b85c2e83f7f8eca9 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf1/75ff11b2f2754de7b85c2e83f7f8eca9 2023-05-04 10:54:11,037 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/9dbc2679fccd43f28085a92704b3ee1d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/cf2/9dbc2679fccd43f28085a92704b3ee1d 2023-05-04 10:54:11,040 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/recovered.edits/90.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369/recovered.edits/90.seqid 2023-05-04 10:54:11,041 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackAndDoubleExecution/ff484ddaba859c44b9223e0332036369 2023-05-04 10:54:11,041 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-05-04 10:54:11,044 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:11,046 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-05-04 10:54:11,047 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-05-04 10:54:11,048 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:11,048 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-05-04 10:54:11,048 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197651048"}]},"ts":"9223372036854775807"} 2023-05-04 10:54:11,050 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-05-04 10:54:11,050 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ff484ddaba859c44b9223e0332036369, NAME => 'testRollbackAndDoubleExecution,,1683197640736.ff484ddaba859c44b9223e0332036369.', STARTKEY => '', ENDKEY => ''}] 2023-05-04 10:54:11,050 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-05-04 10:54:11,050 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683197651050"}]},"ts":"9223372036854775807"} 2023-05-04 10:54:11,052 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-05-04 10:54:11,053 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:54:11,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 30 msec 2023-05-04 10:54:11,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-05-04 10:54:11,128 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-05-04 10:54:11,145 INFO [Listener at localhost/35483] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=401 (was 403), OpenFileDescriptor=838 (was 847), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=319 (was 340), ProcessCount=170 (was 170), AvailableMemoryMB=6446 (was 6539) 2023-05-04 10:54:11,160 INFO [Listener at localhost/35483] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=401, OpenFileDescriptor=838, MaxFileDescriptor=60000, SystemLoadAverage=319, ProcessCount=170, AvailableMemoryMB=6446 2023-05-04 10:54:11,160 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:54:11,162 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-04 10:54:11,163 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-04 10:54:11,164 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:54:11,165 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-05-04 10:54:11,166 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-04 10:54:11,168 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/0def560f25a82ca6fb786e8c5e288761 2023-05-04 10:54:11,168 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/0def560f25a82ca6fb786e8c5e288761 empty. 2023-05-04 10:54:11,169 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/0def560f25a82ca6fb786e8c5e288761 2023-05-04 10:54:11,169 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-05-04 10:54:11,179 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-05-04 10:54:11,180 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7675): creating {ENCODED => 0def560f25a82ca6fb786e8c5e288761, NAME => 'testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp 2023-05-04 10:54:11,188 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:54:11,188 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1604): Closing 0def560f25a82ca6fb786e8c5e288761, disabling compactions & flushes 2023-05-04 10:54:11,188 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761. 2023-05-04 10:54:11,188 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761. 2023-05-04 10:54:11,188 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761. after waiting 0 ms 2023-05-04 10:54:11,188 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761. 2023-05-04 10:54:11,188 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761. 2023-05-04 10:54:11,188 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1558): Region close journal for 0def560f25a82ca6fb786e8c5e288761: 2023-05-04 10:54:11,190 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-05-04 10:54:11,191 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1683197651191"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197651191"}]},"ts":"1683197651191"} 2023-05-04 10:54:11,193 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-04 10:54:11,193 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-04 10:54:11,193 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197651193"}]},"ts":"1683197651193"} 2023-05-04 10:54:11,194 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-05-04 10:54:11,198 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-04 10:54:11,199 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-04 10:54:11,200 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-04 10:54:11,200 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-04 10:54:11,200 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-04 10:54:11,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0def560f25a82ca6fb786e8c5e288761, ASSIGN}] 2023-05-04 10:54:11,201 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0def560f25a82ca6fb786e8c5e288761, ASSIGN 2023-05-04 10:54:11,202 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0def560f25a82ca6fb786e8c5e288761, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42285,1683197625528; forceNewPlan=false, retain=false 2023-05-04 10:54:11,352 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-04 10:54:11,353 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=0def560f25a82ca6fb786e8c5e288761, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:11,354 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1683197651353"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197651353"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197651353"}]},"ts":"1683197651353"} 2023-05-04 10:54:11,355 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 0def560f25a82ca6fb786e8c5e288761, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:54:11,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761. 2023-05-04 10:54:11,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0def560f25a82ca6fb786e8c5e288761, NAME => 'testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761.', STARTKEY => '', ENDKEY => ''} 2023-05-04 10:54:11,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:54:11,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 0def560f25a82ca6fb786e8c5e288761 2023-05-04 10:54:11,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:54:11,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0def560f25a82ca6fb786e8c5e288761 2023-05-04 10:54:11,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0def560f25a82ca6fb786e8c5e288761 2023-05-04 10:54:11,516 INFO [StoreOpener-0def560f25a82ca6fb786e8c5e288761-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0def560f25a82ca6fb786e8c5e288761 2023-05-04 10:54:11,517 DEBUG [StoreOpener-0def560f25a82ca6fb786e8c5e288761-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/0def560f25a82ca6fb786e8c5e288761/cf1 2023-05-04 10:54:11,517 DEBUG [StoreOpener-0def560f25a82ca6fb786e8c5e288761-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/0def560f25a82ca6fb786e8c5e288761/cf1 2023-05-04 10:54:11,518 INFO [StoreOpener-0def560f25a82ca6fb786e8c5e288761-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0def560f25a82ca6fb786e8c5e288761 columnFamilyName cf1 2023-05-04 10:54:11,518 INFO [StoreOpener-0def560f25a82ca6fb786e8c5e288761-1] regionserver.HStore(310): Store=0def560f25a82ca6fb786e8c5e288761/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:54:11,518 INFO [StoreOpener-0def560f25a82ca6fb786e8c5e288761-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0def560f25a82ca6fb786e8c5e288761 2023-05-04 10:54:11,520 DEBUG [StoreOpener-0def560f25a82ca6fb786e8c5e288761-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/0def560f25a82ca6fb786e8c5e288761/cf2 2023-05-04 10:54:11,520 DEBUG [StoreOpener-0def560f25a82ca6fb786e8c5e288761-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/0def560f25a82ca6fb786e8c5e288761/cf2 2023-05-04 10:54:11,521 INFO [StoreOpener-0def560f25a82ca6fb786e8c5e288761-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0def560f25a82ca6fb786e8c5e288761 columnFamilyName cf2 2023-05-04 10:54:11,521 INFO [StoreOpener-0def560f25a82ca6fb786e8c5e288761-1] regionserver.HStore(310): Store=0def560f25a82ca6fb786e8c5e288761/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:54:11,522 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/0def560f25a82ca6fb786e8c5e288761 2023-05-04 10:54:11,523 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/0def560f25a82ca6fb786e8c5e288761 2023-05-04 10:54:11,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:54:11,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0def560f25a82ca6fb786e8c5e288761 2023-05-04 10:54:11,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/0def560f25a82ca6fb786e8c5e288761/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-04 10:54:11,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0def560f25a82ca6fb786e8c5e288761; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63751865, jitterRate=-0.05002318322658539}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:54:11,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0def560f25a82ca6fb786e8c5e288761: 2023-05-04 10:54:11,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761., pid=55, masterSystemTime=1683197651507 2023-05-04 10:54:11,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761. 2023-05-04 10:54:11,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761. 2023-05-04 10:54:11,533 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=0def560f25a82ca6fb786e8c5e288761, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:11,533 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1683197651533"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197651533"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197651533"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197651533"}]},"ts":"1683197651533"} 2023-05-04 10:54:11,536 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-05-04 10:54:11,536 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 0def560f25a82ca6fb786e8c5e288761, server=jenkins-hbase4.apache.org,42285,1683197625528 in 179 msec 2023-05-04 10:54:11,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-05-04 10:54:11,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0def560f25a82ca6fb786e8c5e288761, ASSIGN in 336 msec 2023-05-04 10:54:11,539 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-04 10:54:11,539 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197651539"}]},"ts":"1683197651539"} 2023-05-04 10:54:11,540 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-05-04 10:54:11,542 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-05-04 10:54:11,543 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 379 msec 2023-05-04 10:54:11,598 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-04 10:54:11,665 DEBUG [Listener at localhost/35483] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:11,666 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0def560f25a82ca6fb786e8c5e288761, daughterA=f1f0e2a8ad3e3492a8c308585d3cf77b, daughterB=1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:54:11,667 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0def560f25a82ca6fb786e8c5e288761, daughterA=f1f0e2a8ad3e3492a8c308585d3cf77b, daughterB=1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:54:11,667 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0def560f25a82ca6fb786e8c5e288761, daughterA=f1f0e2a8ad3e3492a8c308585d3cf77b, daughterB=1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:54:11,667 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0def560f25a82ca6fb786e8c5e288761, daughterA=f1f0e2a8ad3e3492a8c308585d3cf77b, daughterB=1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:54:11,669 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0def560f25a82ca6fb786e8c5e288761, UNASSIGN}] 2023-05-04 10:54:11,670 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0def560f25a82ca6fb786e8c5e288761, UNASSIGN 2023-05-04 10:54:11,670 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=0def560f25a82ca6fb786e8c5e288761, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:11,671 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1683197651670"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197651670"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197651670"}]},"ts":"1683197651670"} 2023-05-04 10:54:11,672 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 0def560f25a82ca6fb786e8c5e288761, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:54:11,824 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 0def560f25a82ca6fb786e8c5e288761 2023-05-04 10:54:11,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0def560f25a82ca6fb786e8c5e288761, disabling compactions & flushes 2023-05-04 10:54:11,825 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761. 2023-05-04 10:54:11,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761. 2023-05-04 10:54:11,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761. after waiting 0 ms 2023-05-04 10:54:11,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761. 2023-05-04 10:54:11,829 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/0def560f25a82ca6fb786e8c5e288761/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-05-04 10:54:11,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:54:11,831 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761. 2023-05-04 10:54:11,831 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0def560f25a82ca6fb786e8c5e288761: 2023-05-04 10:54:11,832 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 0def560f25a82ca6fb786e8c5e288761 2023-05-04 10:54:11,836 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=0def560f25a82ca6fb786e8c5e288761, regionState=CLOSED 2023-05-04 10:54:11,836 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1683197651836"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197651836"}]},"ts":"1683197651836"} 2023-05-04 10:54:11,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-05-04 10:54:11,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 0def560f25a82ca6fb786e8c5e288761, server=jenkins-hbase4.apache.org,42285,1683197625528 in 165 msec 2023-05-04 10:54:11,841 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-05-04 10:54:11,841 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0def560f25a82ca6fb786e8c5e288761, UNASSIGN in 170 msec 2023-05-04 10:54:12,270 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/f1f0e2a8ad3e3492a8c308585d3cf77b/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-05-04 10:54:12,272 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/1ba1c1861bf6bdd2c2ab43bc2bebbd9a/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-05-04 10:54:12,274 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1683197652274"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1683197652274"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1683197652274"}]},"ts":"1683197652274"} 2023-05-04 10:54:12,274 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1683197651665.f1f0e2a8ad3e3492a8c308585d3cf77b.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683197652274"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197652274"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197652274"}]},"ts":"1683197652274"} 2023-05-04 10:54:12,274 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1683197651665.1ba1c1861bf6bdd2c2ab43bc2bebbd9a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683197652274"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197652274"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197652274"}]},"ts":"1683197652274"} 2023-05-04 10:54:12,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=f1f0e2a8ad3e3492a8c308585d3cf77b, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1ba1c1861bf6bdd2c2ab43bc2bebbd9a, ASSIGN}] 2023-05-04 10:54:12,278 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1ba1c1861bf6bdd2c2ab43bc2bebbd9a, ASSIGN 2023-05-04 10:54:12,279 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1ba1c1861bf6bdd2c2ab43bc2bebbd9a, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42285,1683197625528; forceNewPlan=false, retain=false 2023-05-04 10:54:12,279 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=f1f0e2a8ad3e3492a8c308585d3cf77b, ASSIGN 2023-05-04 10:54:12,280 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=f1f0e2a8ad3e3492a8c308585d3cf77b, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42285,1683197625528; forceNewPlan=false, retain=false 2023-05-04 10:54:12,429 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-04 10:54:12,431 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=1ba1c1861bf6bdd2c2ab43bc2bebbd9a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:12,431 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1683197651665.1ba1c1861bf6bdd2c2ab43bc2bebbd9a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683197652431"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197652431"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197652431"}]},"ts":"1683197652431"} 2023-05-04 10:54:12,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE; OpenRegionProcedure 1ba1c1861bf6bdd2c2ab43bc2bebbd9a, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:54:12,435 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=f1f0e2a8ad3e3492a8c308585d3cf77b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:12,435 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1683197651665.f1f0e2a8ad3e3492a8c308585d3cf77b.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683197652435"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197652435"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197652435"}]},"ts":"1683197652435"} 2023-05-04 10:54:12,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=59, state=RUNNABLE; OpenRegionProcedure f1f0e2a8ad3e3492a8c308585d3cf77b, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:54:12,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1683197651665.f1f0e2a8ad3e3492a8c308585d3cf77b. 2023-05-04 10:54:12,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f1f0e2a8ad3e3492a8c308585d3cf77b, NAME => 'testSplitTableRegionNoStoreFile,,1683197651665.f1f0e2a8ad3e3492a8c308585d3cf77b.', STARTKEY => '', ENDKEY => '41'} 2023-05-04 10:54:12,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:54:12,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile f1f0e2a8ad3e3492a8c308585d3cf77b 2023-05-04 10:54:12,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1683197651665.f1f0e2a8ad3e3492a8c308585d3cf77b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:54:12,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f1f0e2a8ad3e3492a8c308585d3cf77b 2023-05-04 10:54:12,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f1f0e2a8ad3e3492a8c308585d3cf77b 2023-05-04 10:54:12,589 INFO [StoreOpener-f1f0e2a8ad3e3492a8c308585d3cf77b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f1f0e2a8ad3e3492a8c308585d3cf77b 2023-05-04 10:54:12,591 DEBUG [StoreOpener-f1f0e2a8ad3e3492a8c308585d3cf77b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/f1f0e2a8ad3e3492a8c308585d3cf77b/cf1 2023-05-04 10:54:12,591 DEBUG [StoreOpener-f1f0e2a8ad3e3492a8c308585d3cf77b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/f1f0e2a8ad3e3492a8c308585d3cf77b/cf1 2023-05-04 10:54:12,591 INFO [StoreOpener-f1f0e2a8ad3e3492a8c308585d3cf77b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f1f0e2a8ad3e3492a8c308585d3cf77b columnFamilyName cf1 2023-05-04 10:54:12,592 INFO [StoreOpener-f1f0e2a8ad3e3492a8c308585d3cf77b-1] regionserver.HStore(310): Store=f1f0e2a8ad3e3492a8c308585d3cf77b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:54:12,592 INFO [StoreOpener-f1f0e2a8ad3e3492a8c308585d3cf77b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f1f0e2a8ad3e3492a8c308585d3cf77b 2023-05-04 10:54:12,593 DEBUG [StoreOpener-f1f0e2a8ad3e3492a8c308585d3cf77b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/f1f0e2a8ad3e3492a8c308585d3cf77b/cf2 2023-05-04 10:54:12,593 DEBUG [StoreOpener-f1f0e2a8ad3e3492a8c308585d3cf77b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/f1f0e2a8ad3e3492a8c308585d3cf77b/cf2 2023-05-04 10:54:12,593 INFO [StoreOpener-f1f0e2a8ad3e3492a8c308585d3cf77b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f1f0e2a8ad3e3492a8c308585d3cf77b columnFamilyName cf2 2023-05-04 10:54:12,594 INFO [StoreOpener-f1f0e2a8ad3e3492a8c308585d3cf77b-1] regionserver.HStore(310): Store=f1f0e2a8ad3e3492a8c308585d3cf77b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:54:12,595 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/f1f0e2a8ad3e3492a8c308585d3cf77b 2023-05-04 10:54:12,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/f1f0e2a8ad3e3492a8c308585d3cf77b 2023-05-04 10:54:12,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:54:12,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f1f0e2a8ad3e3492a8c308585d3cf77b 2023-05-04 10:54:12,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f1f0e2a8ad3e3492a8c308585d3cf77b; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75488913, jitterRate=0.12487246096134186}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:54:12,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f1f0e2a8ad3e3492a8c308585d3cf77b: 2023-05-04 10:54:12,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1683197651665.f1f0e2a8ad3e3492a8c308585d3cf77b., pid=62, masterSystemTime=1683197652584 2023-05-04 10:54:12,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1683197651665.f1f0e2a8ad3e3492a8c308585d3cf77b. 2023-05-04 10:54:12,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1683197651665.f1f0e2a8ad3e3492a8c308585d3cf77b. 2023-05-04 10:54:12,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,41,1683197651665.1ba1c1861bf6bdd2c2ab43bc2bebbd9a. 2023-05-04 10:54:12,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1ba1c1861bf6bdd2c2ab43bc2bebbd9a, NAME => 'testSplitTableRegionNoStoreFile,41,1683197651665.1ba1c1861bf6bdd2c2ab43bc2bebbd9a.', STARTKEY => '41', ENDKEY => ''} 2023-05-04 10:54:12,602 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=f1f0e2a8ad3e3492a8c308585d3cf77b, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:12,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:54:12,603 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1683197651665.f1f0e2a8ad3e3492a8c308585d3cf77b.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683197652602"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197652602"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197652602"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197652602"}]},"ts":"1683197652602"} 2023-05-04 10:54:12,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:54:12,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,41,1683197651665.1ba1c1861bf6bdd2c2ab43bc2bebbd9a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:54:12,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:54:12,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:54:12,604 INFO [StoreOpener-1ba1c1861bf6bdd2c2ab43bc2bebbd9a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:54:12,605 DEBUG [StoreOpener-1ba1c1861bf6bdd2c2ab43bc2bebbd9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/1ba1c1861bf6bdd2c2ab43bc2bebbd9a/cf1 2023-05-04 10:54:12,605 DEBUG [StoreOpener-1ba1c1861bf6bdd2c2ab43bc2bebbd9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/1ba1c1861bf6bdd2c2ab43bc2bebbd9a/cf1 2023-05-04 10:54:12,606 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=59 2023-05-04 10:54:12,606 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=59, state=SUCCESS; OpenRegionProcedure f1f0e2a8ad3e3492a8c308585d3cf77b, server=jenkins-hbase4.apache.org,42285,1683197625528 in 167 msec 2023-05-04 10:54:12,606 INFO [StoreOpener-1ba1c1861bf6bdd2c2ab43bc2bebbd9a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1ba1c1861bf6bdd2c2ab43bc2bebbd9a columnFamilyName cf1 2023-05-04 10:54:12,606 INFO [StoreOpener-1ba1c1861bf6bdd2c2ab43bc2bebbd9a-1] regionserver.HStore(310): Store=1ba1c1861bf6bdd2c2ab43bc2bebbd9a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:54:12,607 INFO [StoreOpener-1ba1c1861bf6bdd2c2ab43bc2bebbd9a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:54:12,607 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=f1f0e2a8ad3e3492a8c308585d3cf77b, ASSIGN in 329 msec 2023-05-04 10:54:12,608 DEBUG [StoreOpener-1ba1c1861bf6bdd2c2ab43bc2bebbd9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/1ba1c1861bf6bdd2c2ab43bc2bebbd9a/cf2 2023-05-04 10:54:12,608 DEBUG [StoreOpener-1ba1c1861bf6bdd2c2ab43bc2bebbd9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/1ba1c1861bf6bdd2c2ab43bc2bebbd9a/cf2 2023-05-04 10:54:12,608 INFO [StoreOpener-1ba1c1861bf6bdd2c2ab43bc2bebbd9a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1ba1c1861bf6bdd2c2ab43bc2bebbd9a columnFamilyName cf2 2023-05-04 10:54:12,609 INFO [StoreOpener-1ba1c1861bf6bdd2c2ab43bc2bebbd9a-1] regionserver.HStore(310): Store=1ba1c1861bf6bdd2c2ab43bc2bebbd9a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:54:12,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:54:12,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:54:12,612 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:54:12,613 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:54:12,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1ba1c1861bf6bdd2c2ab43bc2bebbd9a; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70456258, jitterRate=0.04988005757331848}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:54:12,614 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1ba1c1861bf6bdd2c2ab43bc2bebbd9a: 2023-05-04 10:54:12,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1683197651665.1ba1c1861bf6bdd2c2ab43bc2bebbd9a., pid=61, masterSystemTime=1683197652584 2023-05-04 10:54:12,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1683197651665.1ba1c1861bf6bdd2c2ab43bc2bebbd9a. 2023-05-04 10:54:12,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,41,1683197651665.1ba1c1861bf6bdd2c2ab43bc2bebbd9a. 2023-05-04 10:54:12,616 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=1ba1c1861bf6bdd2c2ab43bc2bebbd9a, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:12,616 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1683197651665.1ba1c1861bf6bdd2c2ab43bc2bebbd9a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683197652616"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197652616"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197652616"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197652616"}]},"ts":"1683197652616"} 2023-05-04 10:54:12,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-05-04 10:54:12,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; OpenRegionProcedure 1ba1c1861bf6bdd2c2ab43bc2bebbd9a, server=jenkins-hbase4.apache.org,42285,1683197625528 in 184 msec 2023-05-04 10:54:12,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=56 2023-05-04 10:54:12,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1ba1c1861bf6bdd2c2ab43bc2bebbd9a, ASSIGN in 342 msec 2023-05-04 10:54:12,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0def560f25a82ca6fb786e8c5e288761, daughterA=f1f0e2a8ad3e3492a8c308585d3cf77b, daughterB=1ba1c1861bf6bdd2c2ab43bc2bebbd9a in 955 msec 2023-05-04 10:54:12,670 DEBUG [Listener at localhost/35483] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-04 10:54:12,673 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:54:12,676 INFO [Listener at localhost/35483] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-05-04 10:54:12,677 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-05-04 10:54:12,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:54:12,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-05-04 10:54:12,680 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197652680"}]},"ts":"1683197652680"} 2023-05-04 10:54:12,682 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-05-04 10:54:12,683 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-05-04 10:54:12,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=f1f0e2a8ad3e3492a8c308585d3cf77b, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1ba1c1861bf6bdd2c2ab43bc2bebbd9a, UNASSIGN}] 2023-05-04 10:54:12,685 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1ba1c1861bf6bdd2c2ab43bc2bebbd9a, UNASSIGN 2023-05-04 10:54:12,685 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=1ba1c1861bf6bdd2c2ab43bc2bebbd9a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:12,685 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1683197651665.1ba1c1861bf6bdd2c2ab43bc2bebbd9a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683197652685"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197652685"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197652685"}]},"ts":"1683197652685"} 2023-05-04 10:54:12,687 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 1ba1c1861bf6bdd2c2ab43bc2bebbd9a, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:54:12,689 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=f1f0e2a8ad3e3492a8c308585d3cf77b, UNASSIGN 2023-05-04 10:54:12,690 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=f1f0e2a8ad3e3492a8c308585d3cf77b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:12,690 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1683197651665.f1f0e2a8ad3e3492a8c308585d3cf77b.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683197652690"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197652690"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197652690"}]},"ts":"1683197652690"} 2023-05-04 10:54:12,691 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure f1f0e2a8ad3e3492a8c308585d3cf77b, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:54:12,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-05-04 10:54:12,839 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:54:12,840 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1ba1c1861bf6bdd2c2ab43bc2bebbd9a, disabling compactions & flushes 2023-05-04 10:54:12,840 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,41,1683197651665.1ba1c1861bf6bdd2c2ab43bc2bebbd9a. 2023-05-04 10:54:12,840 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1683197651665.1ba1c1861bf6bdd2c2ab43bc2bebbd9a. 2023-05-04 10:54:12,840 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,41,1683197651665.1ba1c1861bf6bdd2c2ab43bc2bebbd9a. after waiting 0 ms 2023-05-04 10:54:12,840 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,41,1683197651665.1ba1c1861bf6bdd2c2ab43bc2bebbd9a. 2023-05-04 10:54:12,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/1ba1c1861bf6bdd2c2ab43bc2bebbd9a/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-05-04 10:54:12,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:54:12,844 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,41,1683197651665.1ba1c1861bf6bdd2c2ab43bc2bebbd9a. 2023-05-04 10:54:12,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1ba1c1861bf6bdd2c2ab43bc2bebbd9a: 2023-05-04 10:54:12,846 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:54:12,846 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close f1f0e2a8ad3e3492a8c308585d3cf77b 2023-05-04 10:54:12,847 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f1f0e2a8ad3e3492a8c308585d3cf77b, disabling compactions & flushes 2023-05-04 10:54:12,847 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1683197651665.f1f0e2a8ad3e3492a8c308585d3cf77b. 2023-05-04 10:54:12,847 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1683197651665.f1f0e2a8ad3e3492a8c308585d3cf77b. 2023-05-04 10:54:12,847 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1683197651665.f1f0e2a8ad3e3492a8c308585d3cf77b. after waiting 0 ms 2023-05-04 10:54:12,847 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1683197651665.f1f0e2a8ad3e3492a8c308585d3cf77b. 2023-05-04 10:54:12,848 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=1ba1c1861bf6bdd2c2ab43bc2bebbd9a, regionState=CLOSED 2023-05-04 10:54:12,848 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1683197651665.1ba1c1861bf6bdd2c2ab43bc2bebbd9a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683197652847"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197652847"}]},"ts":"1683197652847"} 2023-05-04 10:54:12,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionNoStoreFile/f1f0e2a8ad3e3492a8c308585d3cf77b/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-05-04 10:54:12,852 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-05-04 10:54:12,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:54:12,852 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 1ba1c1861bf6bdd2c2ab43bc2bebbd9a, server=jenkins-hbase4.apache.org,42285,1683197625528 in 162 msec 2023-05-04 10:54:12,852 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1683197651665.f1f0e2a8ad3e3492a8c308585d3cf77b. 2023-05-04 10:54:12,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f1f0e2a8ad3e3492a8c308585d3cf77b: 2023-05-04 10:54:12,854 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed f1f0e2a8ad3e3492a8c308585d3cf77b 2023-05-04 10:54:12,854 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1ba1c1861bf6bdd2c2ab43bc2bebbd9a, UNASSIGN in 169 msec 2023-05-04 10:54:12,854 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=f1f0e2a8ad3e3492a8c308585d3cf77b, regionState=CLOSED 2023-05-04 10:54:12,855 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1683197651665.f1f0e2a8ad3e3492a8c308585d3cf77b.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683197652854"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197652854"}]},"ts":"1683197652854"} 2023-05-04 10:54:12,857 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-05-04 10:54:12,857 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure f1f0e2a8ad3e3492a8c308585d3cf77b, server=jenkins-hbase4.apache.org,42285,1683197625528 in 165 msec 2023-05-04 10:54:12,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=64, resume processing ppid=63 2023-05-04 10:54:12,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=f1f0e2a8ad3e3492a8c308585d3cf77b, UNASSIGN in 173 msec 2023-05-04 10:54:12,859 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197652859"}]},"ts":"1683197652859"} 2023-05-04 10:54:12,860 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-05-04 10:54:12,862 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-05-04 10:54:12,863 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 185 msec 2023-05-04 10:54:12,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-05-04 10:54:12,982 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-05-04 10:54:12,983 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-05-04 10:54:12,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:54:12,985 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:54:12,986 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:54:12,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-05-04 10:54:12,992 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/0def560f25a82ca6fb786e8c5e288761 2023-05-04 10:54:12,992 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:54:12,992 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/f1f0e2a8ad3e3492a8c308585d3cf77b 2023-05-04 10:54:12,994 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/1ba1c1861bf6bdd2c2ab43bc2bebbd9a/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/1ba1c1861bf6bdd2c2ab43bc2bebbd9a/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/1ba1c1861bf6bdd2c2ab43bc2bebbd9a/recovered.edits] 2023-05-04 10:54:12,995 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/f1f0e2a8ad3e3492a8c308585d3cf77b/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/f1f0e2a8ad3e3492a8c308585d3cf77b/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/f1f0e2a8ad3e3492a8c308585d3cf77b/recovered.edits] 2023-05-04 10:54:12,995 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/0def560f25a82ca6fb786e8c5e288761/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/0def560f25a82ca6fb786e8c5e288761/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/0def560f25a82ca6fb786e8c5e288761/recovered.edits] 2023-05-04 10:54:13,002 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/f1f0e2a8ad3e3492a8c308585d3cf77b/recovered.edits/7.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionNoStoreFile/f1f0e2a8ad3e3492a8c308585d3cf77b/recovered.edits/7.seqid 2023-05-04 10:54:13,003 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/1ba1c1861bf6bdd2c2ab43bc2bebbd9a/recovered.edits/7.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionNoStoreFile/1ba1c1861bf6bdd2c2ab43bc2bebbd9a/recovered.edits/7.seqid 2023-05-04 10:54:13,003 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/f1f0e2a8ad3e3492a8c308585d3cf77b 2023-05-04 10:54:13,004 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:54:13,004 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/0def560f25a82ca6fb786e8c5e288761/recovered.edits/4.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionNoStoreFile/0def560f25a82ca6fb786e8c5e288761/recovered.edits/4.seqid 2023-05-04 10:54:13,005 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionNoStoreFile/0def560f25a82ca6fb786e8c5e288761 2023-05-04 10:54:13,005 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-05-04 10:54:13,007 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:54:13,009 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-05-04 10:54:13,011 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-05-04 10:54:13,012 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:54:13,012 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-05-04 10:54:13,012 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197653012"}]},"ts":"9223372036854775807"} 2023-05-04 10:54:13,012 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1683197651665.f1f0e2a8ad3e3492a8c308585d3cf77b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197653012"}]},"ts":"9223372036854775807"} 2023-05-04 10:54:13,012 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1683197651665.1ba1c1861bf6bdd2c2ab43bc2bebbd9a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197653012"}]},"ts":"9223372036854775807"} 2023-05-04 10:54:13,014 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-04 10:54:13,014 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 0def560f25a82ca6fb786e8c5e288761, NAME => 'testSplitTableRegionNoStoreFile,,1683197651163.0def560f25a82ca6fb786e8c5e288761.', STARTKEY => '', ENDKEY => ''}, {ENCODED => f1f0e2a8ad3e3492a8c308585d3cf77b, NAME => 'testSplitTableRegionNoStoreFile,,1683197651665.f1f0e2a8ad3e3492a8c308585d3cf77b.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 1ba1c1861bf6bdd2c2ab43bc2bebbd9a, NAME => 'testSplitTableRegionNoStoreFile,41,1683197651665.1ba1c1861bf6bdd2c2ab43bc2bebbd9a.', STARTKEY => '41', ENDKEY => ''}] 2023-05-04 10:54:13,014 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-05-04 10:54:13,015 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683197653014"}]},"ts":"9223372036854775807"} 2023-05-04 10:54:13,016 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-05-04 10:54:13,018 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:54:13,019 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 35 msec 2023-05-04 10:54:13,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-05-04 10:54:13,088 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-05-04 10:54:13,102 INFO [Listener at localhost/35483] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=407 (was 401) Potentially hanging thread: hconnection-0x2d8012cd-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1612843146_15 at /127.0.0.1:53410 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1612843146_15 at /127.0.0.1:54752 [Waiting for operation #39] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=840 (was 838) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=319 (was 319), ProcessCount=170 (was 170), AvailableMemoryMB=6440 (was 6446) 2023-05-04 10:54:13,116 INFO [Listener at localhost/35483] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=407, OpenFileDescriptor=840, MaxFileDescriptor=60000, SystemLoadAverage=319, ProcessCount=170, AvailableMemoryMB=6439 2023-05-04 10:54:13,116 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:54:13,120 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-04 10:54:13,120 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-04 10:54:13,122 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,124 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-05-04 10:54:13,124 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-04 10:54:13,126 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:13,127 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca empty. 2023-05-04 10:54:13,127 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:13,127 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-05-04 10:54:13,142 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-05-04 10:54:13,143 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6e4db4ef42259e1e3f31375f39c7adca, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp 2023-05-04 10:54:13,157 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:54:13,157 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1604): Closing 6e4db4ef42259e1e3f31375f39c7adca, disabling compactions & flushes 2023-05-04 10:54:13,157 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:13,157 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:13,158 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. after waiting 1 ms 2023-05-04 10:54:13,158 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:13,158 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:13,158 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 6e4db4ef42259e1e3f31375f39c7adca: 2023-05-04 10:54:13,160 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-05-04 10:54:13,161 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1683197653161"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197653161"}]},"ts":"1683197653161"} 2023-05-04 10:54:13,162 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-04 10:54:13,163 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-04 10:54:13,163 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197653163"}]},"ts":"1683197653163"} 2023-05-04 10:54:13,164 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-05-04 10:54:13,167 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-04 10:54:13,167 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-04 10:54:13,167 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-04 10:54:13,168 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-04 10:54:13,168 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-04 10:54:13,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6e4db4ef42259e1e3f31375f39c7adca, ASSIGN}] 2023-05-04 10:54:13,169 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6e4db4ef42259e1e3f31375f39c7adca, ASSIGN 2023-05-04 10:54:13,170 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6e4db4ef42259e1e3f31375f39c7adca, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42285,1683197625528; forceNewPlan=false, retain=false 2023-05-04 10:54:13,320 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-04 10:54:13,322 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=6e4db4ef42259e1e3f31375f39c7adca, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,322 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1683197653321"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197653321"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197653321"}]},"ts":"1683197653321"} 2023-05-04 10:54:13,323 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 6e4db4ef42259e1e3f31375f39c7adca, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:54:13,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:13,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6e4db4ef42259e1e3f31375f39c7adca, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca.', STARTKEY => '', ENDKEY => ''} 2023-05-04 10:54:13,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:54:13,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:13,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:54:13,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:13,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:13,480 INFO [StoreOpener-6e4db4ef42259e1e3f31375f39c7adca-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:13,481 DEBUG [StoreOpener-6e4db4ef42259e1e3f31375f39c7adca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1 2023-05-04 10:54:13,481 DEBUG [StoreOpener-6e4db4ef42259e1e3f31375f39c7adca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1 2023-05-04 10:54:13,482 INFO [StoreOpener-6e4db4ef42259e1e3f31375f39c7adca-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6e4db4ef42259e1e3f31375f39c7adca columnFamilyName cf1 2023-05-04 10:54:13,482 INFO [StoreOpener-6e4db4ef42259e1e3f31375f39c7adca-1] regionserver.HStore(310): Store=6e4db4ef42259e1e3f31375f39c7adca/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:54:13,482 INFO [StoreOpener-6e4db4ef42259e1e3f31375f39c7adca-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:13,484 DEBUG [StoreOpener-6e4db4ef42259e1e3f31375f39c7adca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2 2023-05-04 10:54:13,484 DEBUG [StoreOpener-6e4db4ef42259e1e3f31375f39c7adca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2 2023-05-04 10:54:13,484 INFO [StoreOpener-6e4db4ef42259e1e3f31375f39c7adca-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6e4db4ef42259e1e3f31375f39c7adca columnFamilyName cf2 2023-05-04 10:54:13,485 INFO [StoreOpener-6e4db4ef42259e1e3f31375f39c7adca-1] regionserver.HStore(310): Store=6e4db4ef42259e1e3f31375f39c7adca/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:54:13,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:13,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:13,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:54:13,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:13,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-04 10:54:13,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6e4db4ef42259e1e3f31375f39c7adca; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66270699, jitterRate=-0.01248963177204132}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:54:13,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6e4db4ef42259e1e3f31375f39c7adca: 2023-05-04 10:54:13,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca., pid=71, masterSystemTime=1683197653475 2023-05-04 10:54:13,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:13,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:13,495 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=6e4db4ef42259e1e3f31375f39c7adca, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,496 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1683197653495"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197653495"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197653495"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197653495"}]},"ts":"1683197653495"} 2023-05-04 10:54:13,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-05-04 10:54:13,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 6e4db4ef42259e1e3f31375f39c7adca, server=jenkins-hbase4.apache.org,42285,1683197625528 in 174 msec 2023-05-04 10:54:13,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-05-04 10:54:13,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6e4db4ef42259e1e3f31375f39c7adca, ASSIGN in 331 msec 2023-05-04 10:54:13,502 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-04 10:54:13,502 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197653502"}]},"ts":"1683197653502"} 2023-05-04 10:54:13,503 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-05-04 10:54:13,505 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-05-04 10:54:13,507 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 384 msec 2023-05-04 10:54:13,631 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:54:13,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,632 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:13,632 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:13,632 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-04 10:54:13,633 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-04 10:54:13,633 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,633 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,635 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,635 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:13,635 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:13,635 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:13,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:13,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:13,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:13,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:13,636 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,636 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:54:13,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:13,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:13,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:13,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,637 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:13,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,637 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,637 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,637 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:13,637 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:13,637 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:13,637 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:13,637 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:13,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:13,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,638 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,639 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,639 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,639 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:13,643 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,643 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,644 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:13,644 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:13,644 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:13,644 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:13,644 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:13,644 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:13,644 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:13,644 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:13,644 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:54:13,644 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:54:13,644 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:13,644 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. started... 2023-05-04 10:54:13,645 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6e4db4ef42259e1e3f31375f39c7adca 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-04 10:54:13,646 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,647 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,647 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:13,663 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/6b50d2c64923457dacd08b63dbe0f201 2023-05-04 10:54:13,684 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/a2fba6e70f544183867c0b71a71f8730 2023-05-04 10:54:13,690 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/6b50d2c64923457dacd08b63dbe0f201 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/6b50d2c64923457dacd08b63dbe0f201 2023-05-04 10:54:13,694 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/6b50d2c64923457dacd08b63dbe0f201, entries=2, sequenceid=6, filesize=4.8 K 2023-05-04 10:54:13,695 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/a2fba6e70f544183867c0b71a71f8730 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/a2fba6e70f544183867c0b71a71f8730 2023-05-04 10:54:13,700 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/a2fba6e70f544183867c0b71a71f8730, entries=2, sequenceid=6, filesize=4.8 K 2023-05-04 10:54:13,701 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 6e4db4ef42259e1e3f31375f39c7adca in 56ms, sequenceid=6, compaction requested=false 2023-05-04 10:54:13,701 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:13,701 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6e4db4ef42259e1e3f31375f39c7adca: 2023-05-04 10:54:13,702 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:13,702 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:54:13,702 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:54:13,702 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,702 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:13,702 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:13,703 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,704 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:13,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:13,704 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,704 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:13,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:13,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:13,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:13,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:13,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:13,707 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-04 10:54:13,707 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27187f69[Count = 0] remaining members to acquire global barrier 2023-05-04 10:54:13,707 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-04 10:54:13,707 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,709 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,709 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,709 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,709 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:13,710 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:13,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,710 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:54:13,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,710 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,710 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:54:13,710 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:13,710 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:13,710 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:54:13,710 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:13,710 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:13,710 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:54:13,711 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:13,711 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:13,711 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:13,712 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:13,712 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:13,712 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,712 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:13,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:13,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:13,712 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:13,712 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:13,713 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:13,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:13,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:13,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:13,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:13,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:13,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:13,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:13,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:54:13,717 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:54:13,717 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-04 10:54:13,717 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:54:13,717 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:54:13,717 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,717 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:54:13,719 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:13,719 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,719 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:13,719 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:13,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,719 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:13,719 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,719 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:13,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:13,720 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:13,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:13,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:13,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:13,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:13,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:13,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:13,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,721 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:13,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,723 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:13,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:13,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:13,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:13,725 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:13,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,725 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:13,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:13,726 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:13,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,735 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,735 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,735 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:13,735 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:13,735 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:13,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:13,735 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:13,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:13,735 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:13,735 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:13,735 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:13,735 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:13,735 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:54:13,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:13,735 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:13,735 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:13,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:13,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:13,735 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,735 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:13,736 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,736 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:54:13,736 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:54:13,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:13,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:13,736 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:13,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:13,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:13,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:13,736 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:13,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:13,736 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,736 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,836 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:54:13,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:54:13,849 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:54:13,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-04 10:54:13,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,851 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:13,851 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:13,851 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-04 10:54:13,851 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-04 10:54:13,851 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,851 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,853 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:13,853 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:13,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:13,853 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,853 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:13,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:13,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:13,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:13,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:13,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:13,853 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,853 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:54:13,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,853 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,854 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,854 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:13,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,854 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:13,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:13,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,854 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,854 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,854 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:13,854 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,854 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:13,854 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:13,855 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:13,855 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,854 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:13,855 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:13,855 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:13,855 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,855 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:13,855 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:13,855 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:54:13,855 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:13,855 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:13,855 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:54:13,855 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:13,855 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. started... 2023-05-04 10:54:13,855 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:13,855 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:13,855 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:13,855 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6e4db4ef42259e1e3f31375f39c7adca 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:54:13,855 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:13,855 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:13,857 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,857 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,857 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:13,857 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,858 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,858 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:13,865 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2 2023-05-04 10:54:13,876 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/3728cf3691c143f3a2905a728159a750 2023-05-04 10:54:13,881 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2 2023-05-04 10:54:13,884 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2, entries=10, sequenceid=19, filesize=5.0 K 2023-05-04 10:54:13,885 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/3728cf3691c143f3a2905a728159a750 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/3728cf3691c143f3a2905a728159a750 2023-05-04 10:54:13,889 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/3728cf3691c143f3a2905a728159a750, entries=10, sequenceid=19, filesize=5.0 K 2023-05-04 10:54:13,890 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6e4db4ef42259e1e3f31375f39c7adca in 35ms, sequenceid=19, compaction requested=false 2023-05-04 10:54:13,890 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6e4db4ef42259e1e3f31375f39c7adca: 2023-05-04 10:54:13,890 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:13,890 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:54:13,891 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:54:13,891 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,891 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:13,891 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:13,893 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,893 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:13,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:13,893 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,894 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:13,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:13,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:13,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:13,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:13,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:13,895 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-04 10:54:13,895 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c16bb24[Count = 0] remaining members to acquire global barrier 2023-05-04 10:54:13,895 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-04 10:54:13,895 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,896 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,896 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,896 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,896 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:13,896 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:13,896 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,896 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:54:13,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,896 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:54:13,897 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:13,897 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:13,897 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:54:13,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,897 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:13,897 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:13,897 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:54:13,898 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:13,898 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,898 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:13,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,898 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:13,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:13,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:13,898 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:13,898 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:13,898 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:13,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:13,900 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:13,900 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:13,900 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:13,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:13,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:13,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:13,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:13,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:13,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:13,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:54:13,903 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:54:13,903 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-04 10:54:13,903 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:54:13,903 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:54:13,903 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,903 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:54:13,904 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,904 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:13,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,904 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:13,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:13,904 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,904 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:13,905 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:13,905 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:13,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:13,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,905 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:13,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:13,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:13,905 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:13,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:13,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:13,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:13,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,906 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:13,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:13,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:13,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:13,908 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:13,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,908 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:13,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:13,908 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:13,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,911 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,911 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:13,911 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,911 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:13,911 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:13,911 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:13,911 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:13,911 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:13,911 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:13,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:13,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:13,911 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:13,911 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,911 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:13,911 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:54:13,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:13,911 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:13,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:13,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:13,911 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:13,912 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,912 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:54:13,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:13,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:13,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:13,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:13,912 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:54:13,912 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:13,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:13,912 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:13,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:13,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:13,912 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:13,912 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,012 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:54:14,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:54:14,026 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:54:14,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-04 10:54:14,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,028 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:14,028 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,028 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-04 10:54:14,028 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-04 10:54:14,028 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,029 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,030 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,030 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,030 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,030 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,030 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,030 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:54:14,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:14,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:14,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:14,031 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,031 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,031 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,031 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:14,031 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:14,031 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:14,031 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,031 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,032 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:14,032 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:14,032 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,031 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,032 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:14,032 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,032 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:14,032 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,032 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:14,032 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:14,032 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:14,032 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:14,032 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:54:14,032 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:54:14,032 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:14,032 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. started... 2023-05-04 10:54:14,032 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:14,032 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:14,032 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6e4db4ef42259e1e3f31375f39c7adca 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:54:14,034 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,035 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,035 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,035 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:14,035 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,035 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:14,042 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/e9099b31bca44169b7db92f474f0015f 2023-05-04 10:54:14,053 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf 2023-05-04 10:54:14,057 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/e9099b31bca44169b7db92f474f0015f as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/e9099b31bca44169b7db92f474f0015f 2023-05-04 10:54:14,060 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/e9099b31bca44169b7db92f474f0015f, entries=10, sequenceid=32, filesize=5.0 K 2023-05-04 10:54:14,061 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf 2023-05-04 10:54:14,065 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf, entries=10, sequenceid=32, filesize=5.0 K 2023-05-04 10:54:14,066 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6e4db4ef42259e1e3f31375f39c7adca in 34ms, sequenceid=32, compaction requested=true 2023-05-04 10:54:14,066 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6e4db4ef42259e1e3f31375f39c7adca: 2023-05-04 10:54:14,066 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:14,066 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:54:14,066 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:54:14,066 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,066 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:14,066 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:14,070 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,070 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:14,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:14,071 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:14,071 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:14,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:14,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:14,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-04 10:54:14,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4524eff5[Count = 0] remaining members to acquire global barrier 2023-05-04 10:54:14,073 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-04 10:54:14,073 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,075 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,075 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,075 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,075 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:14,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,076 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:14,076 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:14,075 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,076 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:54:14,076 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:54:14,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,075 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:14,076 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:54:14,076 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:14,076 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:14,076 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:54:14,077 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:14,077 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,077 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:14,078 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:14,078 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,078 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:14,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:14,078 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:14,078 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:14,078 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,078 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:14,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:14,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:14,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:14,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:54:14,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:54:14,081 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-04 10:54:14,081 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:54:14,081 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:54:14,081 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,081 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:54:14,082 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,082 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:14,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:14,082 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,082 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:14,082 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,083 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,082 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,082 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,083 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:14,084 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:14,085 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,086 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,086 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,090 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,090 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,090 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,090 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,090 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,090 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,090 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,090 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,090 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,090 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:14,090 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:54:14,090 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,090 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,090 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:54:14,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,090 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:54:14,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,090 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,091 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,091 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,091 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,091 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,091 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,091 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,191 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:54:14,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:54:14,204 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:54:14,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-04 10:54:14,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,206 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:14,206 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,206 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-04 10:54:14,206 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-04 10:54:14,206 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,206 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,207 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,207 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,207 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,207 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,208 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,208 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:54:14,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,208 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,208 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,208 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,209 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,209 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,209 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,209 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,209 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:14,209 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,210 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,210 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:14,210 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:14,210 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:14,210 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,210 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:14,210 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:14,210 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,210 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,210 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:14,210 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:14,210 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:14,210 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:14,210 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:14,210 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:54:14,210 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:54:14,210 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:14,210 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. started... 2023-05-04 10:54:14,210 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6e4db4ef42259e1e3f31375f39c7adca 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:54:14,211 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,211 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,212 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,212 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:14,212 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,212 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:14,223 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/4e1aa321a8e54e6a89b09587a5002040 2023-05-04 10:54:14,239 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/314d84ca037f4e72a11fe4feda85e8e6 2023-05-04 10:54:14,244 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/4e1aa321a8e54e6a89b09587a5002040 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/4e1aa321a8e54e6a89b09587a5002040 2023-05-04 10:54:14,248 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/4e1aa321a8e54e6a89b09587a5002040, entries=10, sequenceid=45, filesize=5.0 K 2023-05-04 10:54:14,249 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/314d84ca037f4e72a11fe4feda85e8e6 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/314d84ca037f4e72a11fe4feda85e8e6 2023-05-04 10:54:14,252 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/314d84ca037f4e72a11fe4feda85e8e6, entries=10, sequenceid=45, filesize=5.0 K 2023-05-04 10:54:14,253 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6e4db4ef42259e1e3f31375f39c7adca in 43ms, sequenceid=45, compaction requested=true 2023-05-04 10:54:14,254 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6e4db4ef42259e1e3f31375f39c7adca: 2023-05-04 10:54:14,254 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:14,254 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:54:14,254 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:54:14,254 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,254 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:14,254 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:14,255 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,255 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:14,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:14,256 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,256 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:14,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:14,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:14,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:14,257 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-04 10:54:14,257 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ab66093[Count = 0] remaining members to acquire global barrier 2023-05-04 10:54:14,258 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-04 10:54:14,258 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,259 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,259 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,259 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,260 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,260 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:14,260 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:14,260 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:54:14,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,260 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:54:14,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,260 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:14,260 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:14,260 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:54:14,260 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:14,260 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:14,260 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:54:14,261 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:14,261 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,261 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:14,262 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,262 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:14,262 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:14,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:14,262 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:14,263 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:14,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:14,263 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,263 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:14,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:14,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:14,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:54:14,266 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:54:14,266 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-04 10:54:14,266 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:54:14,266 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:54:14,266 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,266 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:54:14,271 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,271 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:14,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:14,271 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,271 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,271 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,271 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:14,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,271 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,272 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,272 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:14,273 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:14,274 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,275 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,275 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,278 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,278 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,278 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,278 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,278 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,278 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,278 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,278 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,278 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,278 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:14,278 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:54:14,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,278 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,278 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,279 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,279 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,279 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:54:14,279 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:54:14,279 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,279 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,279 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,279 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,279 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,379 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:54:14,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:54:14,393 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:54:14,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-04 10:54:14,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,394 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:14,394 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,394 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-04 10:54:14,394 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-04 10:54:14,395 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,395 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,396 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,396 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,396 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,396 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,397 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,397 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:54:14,397 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,397 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,397 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:14,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:14,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:14,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,397 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,397 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,397 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,398 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:14,398 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,398 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:14,398 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:14,398 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,398 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,398 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:14,398 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:14,398 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,398 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:14,398 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:14,398 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,398 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:14,398 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:14,398 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,398 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:14,398 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:14,398 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:14,398 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:14,398 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:14,398 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:54:14,398 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:54:14,398 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. started... 2023-05-04 10:54:14,398 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6e4db4ef42259e1e3f31375f39c7adca 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:54:14,400 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,402 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,402 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,402 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:14,402 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,402 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:14,410 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/f8f24e978d664e0bae7ec180d57833b9 2023-05-04 10:54:14,423 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/65eaa2b193064baea11fcdb4a27b3a9d 2023-05-04 10:54:14,427 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/f8f24e978d664e0bae7ec180d57833b9 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/f8f24e978d664e0bae7ec180d57833b9 2023-05-04 10:54:14,431 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/f8f24e978d664e0bae7ec180d57833b9, entries=10, sequenceid=58, filesize=5.0 K 2023-05-04 10:54:14,431 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/65eaa2b193064baea11fcdb4a27b3a9d as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/65eaa2b193064baea11fcdb4a27b3a9d 2023-05-04 10:54:14,435 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/65eaa2b193064baea11fcdb4a27b3a9d, entries=10, sequenceid=58, filesize=5.0 K 2023-05-04 10:54:14,435 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6e4db4ef42259e1e3f31375f39c7adca in 37ms, sequenceid=58, compaction requested=true 2023-05-04 10:54:14,436 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6e4db4ef42259e1e3f31375f39c7adca: 2023-05-04 10:54:14,436 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:14,436 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:54:14,436 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:54:14,436 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,436 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:14,436 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:14,437 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,437 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:14,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:14,438 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,438 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:14,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:14,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:14,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:14,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-04 10:54:14,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3eb1bf9b[Count = 0] remaining members to acquire global barrier 2023-05-04 10:54:14,440 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-04 10:54:14,440 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,441 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,441 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,441 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,441 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:14,441 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:14,441 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,441 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:54:14,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,441 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:14,441 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:14,441 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:14,441 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:54:14,441 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:54:14,441 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:14,441 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:54:14,442 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:14,442 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,442 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,442 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:14,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:14,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:14,442 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:14,443 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,443 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:14,444 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:14,444 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:14,444 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:14,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:14,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:14,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:54:14,447 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:54:14,447 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-04 10:54:14,447 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:54:14,447 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:54:14,447 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,447 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:54:14,448 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,448 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,448 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:14,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:14,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,448 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,448 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,448 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:14,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,449 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,449 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,449 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,454 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:14,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:14,456 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,456 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,457 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,459 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,459 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,459 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,459 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,459 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:14,460 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:54:14,459 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,460 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,459 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,460 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:54:14,459 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,459 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,460 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,460 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,460 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:54:14,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,460 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,460 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,461 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,461 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,461 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,461 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,461 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,560 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:54:14,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:54:14,575 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:54:14,576 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-04 10:54:14,576 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,577 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:14,577 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,577 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-04 10:54:14,577 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-04 10:54:14,577 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,577 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,582 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,582 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,582 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,582 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,582 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,582 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:54:14,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,583 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,583 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,583 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:14,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:14,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,583 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,583 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:14,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,584 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,584 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:14,584 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,584 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:14,584 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,584 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:14,584 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:14,584 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:14,584 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:14,584 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:14,584 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,584 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,584 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:14,584 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,584 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:14,584 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,584 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:14,584 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:14,584 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:14,585 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:14,585 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:54:14,585 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:54:14,585 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:14,585 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. started... 2023-05-04 10:54:14,585 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6e4db4ef42259e1e3f31375f39c7adca 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:54:14,585 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,586 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,586 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:14,587 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,587 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,587 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:14,599 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/4d11ba96a1c4419ab7fc92485270da71 2023-05-04 10:54:14,616 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/2bc73271d81c4c30aa4473b1ed7ca1c5 2023-05-04 10:54:14,620 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/4d11ba96a1c4419ab7fc92485270da71 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/4d11ba96a1c4419ab7fc92485270da71 2023-05-04 10:54:14,625 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/4d11ba96a1c4419ab7fc92485270da71, entries=10, sequenceid=71, filesize=5.0 K 2023-05-04 10:54:14,626 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/2bc73271d81c4c30aa4473b1ed7ca1c5 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/2bc73271d81c4c30aa4473b1ed7ca1c5 2023-05-04 10:54:14,630 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/2bc73271d81c4c30aa4473b1ed7ca1c5, entries=10, sequenceid=71, filesize=5.0 K 2023-05-04 10:54:14,631 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6e4db4ef42259e1e3f31375f39c7adca in 45ms, sequenceid=71, compaction requested=true 2023-05-04 10:54:14,631 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6e4db4ef42259e1e3f31375f39c7adca: 2023-05-04 10:54:14,631 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:14,631 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:54:14,631 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:54:14,631 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,631 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:14,631 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:14,633 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,633 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,633 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:14,634 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:14,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:14,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:14,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:14,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:14,636 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-04 10:54:14,636 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c7fcf2[Count = 0] remaining members to acquire global barrier 2023-05-04 10:54:14,636 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-04 10:54:14,636 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,638 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,638 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,638 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,639 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:14,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,639 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:14,639 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,639 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:54:14,639 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:14,639 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:14,639 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:54:14,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,639 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:54:14,639 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:14,639 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:14,639 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:54:14,641 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:14,641 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:14,641 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,641 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,641 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:14,641 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:14,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:14,641 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:14,641 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,641 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:14,641 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:14,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:14,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:14,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:14,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:54:14,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:54:14,645 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-04 10:54:14,645 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:54:14,645 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:54:14,645 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,645 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:54:14,648 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,648 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,648 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,648 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,648 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,648 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,648 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,648 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:14,649 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,649 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:14,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,651 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:14,651 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,651 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,654 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,654 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,654 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,654 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,654 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,654 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:14,654 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,654 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,654 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,654 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,655 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,654 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:54:14,655 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,654 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,654 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,655 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,655 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,655 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,655 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:54:14,655 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,655 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:54:14,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,655 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,755 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:54:14,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:54:14,769 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:54:14,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-04 10:54:14,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,771 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:14,771 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,771 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-04 10:54:14,771 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-04 10:54:14,771 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,771 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,773 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,773 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,773 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,773 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,774 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,774 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:54:14,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,774 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,774 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,775 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:14,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,775 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:14,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,775 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:14,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:14,775 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,775 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,775 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,775 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:14,775 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,775 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:14,775 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:14,775 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,775 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:14,775 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,775 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:54:14,776 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:14,776 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:14,775 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:14,776 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. started... 2023-05-04 10:54:14,776 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,776 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:54:14,776 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:14,776 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6e4db4ef42259e1e3f31375f39c7adca 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-05-04 10:54:14,776 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,776 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:14,776 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:14,776 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:14,776 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:14,776 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:14,777 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,778 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,778 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:14,778 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,779 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,779 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:14,786 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/0a68f3a23f2b4f22a848d740c34ee4b7 2023-05-04 10:54:14,789 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a68f3a23f2b4f22a848d740c34ee4b7 2023-05-04 10:54:14,797 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/4e78c3f056a94e26b2e7a35b0572da21 2023-05-04 10:54:14,800 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4e78c3f056a94e26b2e7a35b0572da21 2023-05-04 10:54:14,801 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/0a68f3a23f2b4f22a848d740c34ee4b7 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/0a68f3a23f2b4f22a848d740c34ee4b7 2023-05-04 10:54:14,805 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a68f3a23f2b4f22a848d740c34ee4b7 2023-05-04 10:54:14,805 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/0a68f3a23f2b4f22a848d740c34ee4b7, entries=9, sequenceid=83, filesize=5.1 K 2023-05-04 10:54:14,806 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/4e78c3f056a94e26b2e7a35b0572da21 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/4e78c3f056a94e26b2e7a35b0572da21 2023-05-04 10:54:14,809 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4e78c3f056a94e26b2e7a35b0572da21 2023-05-04 10:54:14,809 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/4e78c3f056a94e26b2e7a35b0572da21, entries=9, sequenceid=83, filesize=5.1 K 2023-05-04 10:54:14,810 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 6e4db4ef42259e1e3f31375f39c7adca in 34ms, sequenceid=83, compaction requested=true 2023-05-04 10:54:14,810 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6e4db4ef42259e1e3f31375f39c7adca: 2023-05-04 10:54:14,810 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:14,810 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:54:14,810 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:54:14,810 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,810 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:14,810 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:14,812 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,812 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:14,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:14,812 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,812 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:14,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:14,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:14,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:14,814 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-04 10:54:14,814 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@65fee0b8[Count = 0] remaining members to acquire global barrier 2023-05-04 10:54:14,814 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-04 10:54:14,814 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,816 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,816 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,816 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,816 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,816 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:14,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,816 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:14,816 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:14,816 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:54:14,816 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:54:14,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,816 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:14,817 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:14,817 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:14,817 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:54:14,817 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:54:14,818 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:14,818 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,818 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,818 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:14,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:14,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:14,821 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:14,821 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,821 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:14,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:14,823 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:14,823 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,823 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:14,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:14,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:14,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:54:14,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:54:14,825 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-04 10:54:14,825 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:54:14,826 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:54:14,826 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,826 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:54:14,827 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,827 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,827 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,827 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,827 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,827 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,827 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,828 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:14,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:14,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,828 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:14,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,829 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:14,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,830 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:14,830 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,831 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,833 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,833 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,833 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,833 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,833 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,833 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,833 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,833 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:14,833 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:54:14,833 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,833 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,833 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,834 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:54:14,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,833 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,834 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:54:14,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,834 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:14,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,834 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,834 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,834 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,834 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:14,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:14,834 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,834 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,934 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:54:14,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:54:14,943 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:54:14,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-04 10:54:14,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,945 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:14,945 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,945 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-04 10:54:14,945 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-04 10:54:14,945 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,945 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,946 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,946 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,946 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,946 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:14,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:14,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:14,947 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,947 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:54:14,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,947 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,947 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,947 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:14,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:14,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:14,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,948 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,948 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,948 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,948 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:14,948 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:14,948 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,948 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:14,948 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,948 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,948 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:14,948 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:14,948 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,948 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:14,948 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:14,949 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:14,949 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:14,948 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,949 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:14,949 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:14,949 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:14,949 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:14,949 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:14,949 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:54:14,949 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:54:14,949 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:14,949 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. started... 2023-05-04 10:54:14,949 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6e4db4ef42259e1e3f31375f39c7adca 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-05-04 10:54:14,951 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,952 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,952 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,952 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:14,952 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,952 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:14,962 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/83e29b0da26a44bf9dee17b57dc4bfb0 2023-05-04 10:54:14,966 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 83e29b0da26a44bf9dee17b57dc4bfb0 2023-05-04 10:54:14,974 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/70eb3fe40ebf40349a58cd4c168f2344 2023-05-04 10:54:14,978 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 70eb3fe40ebf40349a58cd4c168f2344 2023-05-04 10:54:14,979 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/83e29b0da26a44bf9dee17b57dc4bfb0 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/83e29b0da26a44bf9dee17b57dc4bfb0 2023-05-04 10:54:14,983 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 83e29b0da26a44bf9dee17b57dc4bfb0 2023-05-04 10:54:14,983 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/83e29b0da26a44bf9dee17b57dc4bfb0, entries=5, sequenceid=91, filesize=5.0 K 2023-05-04 10:54:14,984 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/70eb3fe40ebf40349a58cd4c168f2344 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/70eb3fe40ebf40349a58cd4c168f2344 2023-05-04 10:54:14,991 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 70eb3fe40ebf40349a58cd4c168f2344 2023-05-04 10:54:14,991 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/70eb3fe40ebf40349a58cd4c168f2344, entries=5, sequenceid=91, filesize=5.0 K 2023-05-04 10:54:14,992 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 6e4db4ef42259e1e3f31375f39c7adca in 43ms, sequenceid=91, compaction requested=true 2023-05-04 10:54:14,992 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6e4db4ef42259e1e3f31375f39c7adca: 2023-05-04 10:54:14,992 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:14,992 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:54:14,992 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:54:14,992 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,992 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:14,992 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:14,993 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,993 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:14,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:14,994 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,994 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:14,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:14,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:14,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:14,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:14,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:14,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-04 10:54:14,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ca5a66c[Count = 0] remaining members to acquire global barrier 2023-05-04 10:54:14,996 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-04 10:54:14,996 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,997 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,997 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,997 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,997 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:14,997 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:14,997 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:54:14,997 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:14,997 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:54:14,997 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:14,997 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:14,997 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:54:14,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:14,997 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:14,997 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:14,997 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:54:14,999 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:14,999 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:14,999 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:15,000 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:15,000 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,000 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:15,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,000 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:15,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:15,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:15,000 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:15,000 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:15,000 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:15,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:15,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:15,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:15,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:15,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:15,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:15,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:15,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:54:15,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:54:15,003 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-04 10:54:15,003 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:54:15,003 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:54:15,003 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,003 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:54:15,005 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,005 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,005 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:15,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:15,005 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:15,005 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:15,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,005 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:15,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:15,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:15,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:15,005 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:15,005 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:15,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:15,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:15,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:15,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,006 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:15,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,007 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:15,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:15,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:15,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:15,008 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:15,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,008 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:15,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:15,009 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:15,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,015 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,015 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:15,015 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:15,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:15,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:15,015 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:15,015 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,015 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:15,015 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:15,015 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:15,015 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:15,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:15,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:15,015 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:15,015 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:54:15,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:15,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:15,015 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:15,015 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:15,015 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,016 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,016 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:15,016 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:54:15,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:15,016 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:54:15,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:15,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:15,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:15,016 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:15,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:15,016 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:15,016 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,016 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,116 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:54:15,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:54:15,125 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:54:15,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-04 10:54:15,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,126 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:15,126 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:15,126 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-04 10:54:15,126 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-04 10:54:15,127 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,127 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,128 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,128 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:15,128 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:15,128 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:15,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:15,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:15,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:15,128 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,128 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:54:15,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:15,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:15,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:15,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,129 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,129 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,129 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:15,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,129 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:15,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:54:15,130 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:15,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,130 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,130 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:15,130 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:15,130 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:15,130 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:15,130 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-04 10:54:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:15,130 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-04 10:54:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:54:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-04 10:54:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:15,132 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,132 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,132 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,132 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:15,132 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,132 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:15,133 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:54:15,133 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:54:15,133 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:15,133 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. started... 2023-05-04 10:54:15,133 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6e4db4ef42259e1e3f31375f39c7adca 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-05-04 10:54:15,142 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/aa66485bbe564c6da61e1b86de6ab481 2023-05-04 10:54:15,146 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aa66485bbe564c6da61e1b86de6ab481 2023-05-04 10:54:15,157 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/60f398e4d8c541c897fd562fd3ad73c1 2023-05-04 10:54:15,161 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 60f398e4d8c541c897fd562fd3ad73c1 2023-05-04 10:54:15,162 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/aa66485bbe564c6da61e1b86de6ab481 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/aa66485bbe564c6da61e1b86de6ab481 2023-05-04 10:54:15,165 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aa66485bbe564c6da61e1b86de6ab481 2023-05-04 10:54:15,165 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/aa66485bbe564c6da61e1b86de6ab481, entries=5, sequenceid=99, filesize=5.0 K 2023-05-04 10:54:15,166 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/60f398e4d8c541c897fd562fd3ad73c1 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/60f398e4d8c541c897fd562fd3ad73c1 2023-05-04 10:54:15,169 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 60f398e4d8c541c897fd562fd3ad73c1 2023-05-04 10:54:15,170 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/60f398e4d8c541c897fd562fd3ad73c1, entries=5, sequenceid=99, filesize=5.0 K 2023-05-04 10:54:15,170 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 6e4db4ef42259e1e3f31375f39c7adca in 37ms, sequenceid=99, compaction requested=true 2023-05-04 10:54:15,170 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6e4db4ef42259e1e3f31375f39c7adca: 2023-05-04 10:54:15,170 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:15,171 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:54:15,171 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:54:15,171 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,171 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-04 10:54:15,171 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-04 10:54:15,172 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,172 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:15,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:15,173 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,173 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:54:15,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:15,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:15,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:15,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:15,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:15,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-04 10:54:15,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3fdb9d04[Count = 0] remaining members to acquire global barrier 2023-05-04 10:54:15,174 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-04 10:54:15,174 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,177 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,177 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,177 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,177 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:15,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,177 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:15,177 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-04 10:54:15,177 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:15,177 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:15,177 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,177 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:54:15,177 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:54:15,177 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:54:15,177 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-04 10:54:15,177 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:54:15,178 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:15,178 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:15,178 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:15,178 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:15,178 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:15,178 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,178 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:15,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:15,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:15,179 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-04 10:54:15,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:15,179 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:15,179 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-04 10:54:15,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:15,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:15,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:15,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:15,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:15,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:15,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:54:15,182 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:54:15,182 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-04 10:54:15,182 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:54:15,182 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:54:15,182 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,182 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:54:15,184 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,184 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,184 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:54:15,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:54:15,184 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:15,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,184 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:15,184 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:15,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:54:15,184 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:15,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:15,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:15,184 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:15,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:15,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,185 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:15,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:15,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:15,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:54:15,185 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:15,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:15,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,187 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:15,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:54:15,187 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:15,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,187 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:15,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:15,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,190 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,191 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,191 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:15,191 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:15,191 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:54:15,191 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:54:15,191 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-04 10:54:15,191 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:15,191 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:54:15,191 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:15,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:15,191 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:54:15,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:15,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:15,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:15,191 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:54:15,191 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:15,191 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:54:15,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:54:15,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:15,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:15,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:15,191 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:15,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:15,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:54:15,192 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,192 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,192 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,192 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:54:15,192 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:15,192 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,192 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:15,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:54:15,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:54:15,291 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:54:15,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:54:15,295 DEBUG [Listener at localhost/35483] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,296 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=6e4db4ef42259e1e3f31375f39c7adca, daughterA=4bb4acbdcedf2f8531e747ec59b78d24, daughterB=307b832391c13214298b00417783c5e0 2023-05-04 10:54:15,297 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=6e4db4ef42259e1e3f31375f39c7adca, daughterA=4bb4acbdcedf2f8531e747ec59b78d24, daughterB=307b832391c13214298b00417783c5e0 2023-05-04 10:54:15,297 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=6e4db4ef42259e1e3f31375f39c7adca, daughterA=4bb4acbdcedf2f8531e747ec59b78d24, daughterB=307b832391c13214298b00417783c5e0 2023-05-04 10:54:15,297 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=6e4db4ef42259e1e3f31375f39c7adca, daughterA=4bb4acbdcedf2f8531e747ec59b78d24, daughterB=307b832391c13214298b00417783c5e0 2023-05-04 10:54:15,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6e4db4ef42259e1e3f31375f39c7adca, UNASSIGN}] 2023-05-04 10:54:15,299 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6e4db4ef42259e1e3f31375f39c7adca, UNASSIGN 2023-05-04 10:54:15,300 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=6e4db4ef42259e1e3f31375f39c7adca, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,300 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1683197655300"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197655300"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197655300"}]},"ts":"1683197655300"} 2023-05-04 10:54:15,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 6e4db4ef42259e1e3f31375f39c7adca, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:54:15,453 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6e4db4ef42259e1e3f31375f39c7adca, disabling compactions & flushes 2023-05-04 10:54:15,453 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:15,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:15,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. after waiting 0 ms 2023-05-04 10:54:15,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:15,453 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 6e4db4ef42259e1e3f31375f39c7adca 2/2 column families, dataSize=50 B heapSize=720 B 2023-05-04 10:54:15,463 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/96848d837e2f45e098225129736df1e9 2023-05-04 10:54:15,466 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 96848d837e2f45e098225129736df1e9 2023-05-04 10:54:15,475 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/315c805320ed452280fc907b4c3b8204 2023-05-04 10:54:15,478 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 315c805320ed452280fc907b4c3b8204 2023-05-04 10:54:15,479 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf1/96848d837e2f45e098225129736df1e9 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/96848d837e2f45e098225129736df1e9 2023-05-04 10:54:15,483 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 96848d837e2f45e098225129736df1e9 2023-05-04 10:54:15,483 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/96848d837e2f45e098225129736df1e9, entries=1, sequenceid=103, filesize=4.9 K 2023-05-04 10:54:15,484 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.tmp/cf2/315c805320ed452280fc907b4c3b8204 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/315c805320ed452280fc907b4c3b8204 2023-05-04 10:54:15,487 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 315c805320ed452280fc907b4c3b8204 2023-05-04 10:54:15,487 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/315c805320ed452280fc907b4c3b8204, entries=1, sequenceid=103, filesize=4.9 K 2023-05-04 10:54:15,488 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 6e4db4ef42259e1e3f31375f39c7adca in 35ms, sequenceid=103, compaction requested=true 2023-05-04 10:54:15,493 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-05-04 10:54:15,493 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:54:15,493 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca. 2023-05-04 10:54:15,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6e4db4ef42259e1e3f31375f39c7adca: 2023-05-04 10:54:15,495 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,495 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=6e4db4ef42259e1e3f31375f39c7adca, regionState=CLOSED 2023-05-04 10:54:15,495 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1683197655495"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197655495"}]},"ts":"1683197655495"} 2023-05-04 10:54:15,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-05-04 10:54:15,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 6e4db4ef42259e1e3f31375f39c7adca, server=jenkins-hbase4.apache.org,42285,1683197625528 in 196 msec 2023-05-04 10:54:15,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-05-04 10:54:15,500 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6e4db4ef42259e1e3f31375f39c7adca, UNASSIGN in 199 msec 2023-05-04 10:54:15,514 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=6e4db4ef42259e1e3f31375f39c7adca, threads=16 2023-05-04 10:54:15,514 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/2bc73271d81c4c30aa4473b1ed7ca1c5 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,516 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/314d84ca037f4e72a11fe4feda85e8e6 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,516 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/315c805320ed452280fc907b4c3b8204 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,517 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/3728cf3691c143f3a2905a728159a750 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,523 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/4e78c3f056a94e26b2e7a35b0572da21 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,524 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/60f398e4d8c541c897fd562fd3ad73c1 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,524 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/65eaa2b193064baea11fcdb4a27b3a9d for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,524 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/70eb3fe40ebf40349a58cd4c168f2344 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,524 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/a2fba6e70f544183867c0b71a71f8730 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,525 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,527 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/2bc73271d81c4c30aa4473b1ed7ca1c5, top=false 2023-05-04 10:54:15,528 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/0a68f3a23f2b4f22a848d740c34ee4b7 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,530 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 315c805320ed452280fc907b4c3b8204 2023-05-04 10:54:15,530 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/314d84ca037f4e72a11fe4feda85e8e6, top=false 2023-05-04 10:54:15,532 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,535 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/4d11ba96a1c4419ab7fc92485270da71 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,535 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/4e1aa321a8e54e6a89b09587a5002040 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,536 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/6b50d2c64923457dacd08b63dbe0f201 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,540 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/83e29b0da26a44bf9dee17b57dc4bfb0 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,543 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/65eaa2b193064baea11fcdb4a27b3a9d, top=false 2023-05-04 10:54:15,545 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 70eb3fe40ebf40349a58cd4c168f2344 2023-05-04 10:54:15,550 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 60f398e4d8c541c897fd562fd3ad73c1 2023-05-04 10:54:15,552 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 315c805320ed452280fc907b4c3b8204 2023-05-04 10:54:15,552 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/315c805320ed452280fc907b4c3b8204, top=true 2023-05-04 10:54:15,553 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 70eb3fe40ebf40349a58cd4c168f2344 2023-05-04 10:54:15,553 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/70eb3fe40ebf40349a58cd4c168f2344, top=true 2023-05-04 10:54:15,554 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4e78c3f056a94e26b2e7a35b0572da21 2023-05-04 10:54:15,556 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a68f3a23f2b4f22a848d740c34ee4b7 2023-05-04 10:54:15,558 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 83e29b0da26a44bf9dee17b57dc4bfb0 2023-05-04 10:54:15,562 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/4d11ba96a1c4419ab7fc92485270da71, top=false 2023-05-04 10:54:15,567 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 83e29b0da26a44bf9dee17b57dc4bfb0 2023-05-04 10:54:15,567 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/83e29b0da26a44bf9dee17b57dc4bfb0, top=true 2023-05-04 10:54:15,571 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 60f398e4d8c541c897fd562fd3ad73c1 2023-05-04 10:54:15,571 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/60f398e4d8c541c897fd562fd3ad73c1, top=true 2023-05-04 10:54:15,572 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/4e1aa321a8e54e6a89b09587a5002040, top=false 2023-05-04 10:54:15,588 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.splits/4bb4acbdcedf2f8531e747ec59b78d24/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-2bc73271d81c4c30aa4473b1ed7ca1c5 for child: 4bb4acbdcedf2f8531e747ec59b78d24, parent: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,597 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.splits/4bb4acbdcedf2f8531e747ec59b78d24/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-314d84ca037f4e72a11fe4feda85e8e6 for child: 4bb4acbdcedf2f8531e747ec59b78d24, parent: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,601 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.splits/4bb4acbdcedf2f8531e747ec59b78d24/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-65eaa2b193064baea11fcdb4a27b3a9d for child: 4bb4acbdcedf2f8531e747ec59b78d24, parent: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,605 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/314d84ca037f4e72a11fe4feda85e8e6 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,605 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/96848d837e2f45e098225129736df1e9 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,606 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/65eaa2b193064baea11fcdb4a27b3a9d for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,606 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/aa66485bbe564c6da61e1b86de6ab481 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,608 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aa66485bbe564c6da61e1b86de6ab481 2023-05-04 10:54:15,611 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aa66485bbe564c6da61e1b86de6ab481 2023-05-04 10:54:15,611 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/aa66485bbe564c6da61e1b86de6ab481, top=true 2023-05-04 10:54:15,613 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.splits/307b832391c13214298b00417783c5e0/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-70eb3fe40ebf40349a58cd4c168f2344 for child: 307b832391c13214298b00417783c5e0, parent: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,613 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/70eb3fe40ebf40349a58cd4c168f2344 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,613 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/e9099b31bca44169b7db92f474f0015f for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,613 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.splits/4bb4acbdcedf2f8531e747ec59b78d24/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-4d11ba96a1c4419ab7fc92485270da71 for child: 4bb4acbdcedf2f8531e747ec59b78d24, parent: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,616 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.splits/307b832391c13214298b00417783c5e0/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-315c805320ed452280fc907b4c3b8204 for child: 307b832391c13214298b00417783c5e0, parent: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,616 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/315c805320ed452280fc907b4c3b8204 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,616 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/f8f24e978d664e0bae7ec180d57833b9 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,619 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/f8f24e978d664e0bae7ec180d57833b9, top=false 2023-05-04 10:54:15,628 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.splits/307b832391c13214298b00417783c5e0/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-60f398e4d8c541c897fd562fd3ad73c1 for child: 307b832391c13214298b00417783c5e0, parent: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,628 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/60f398e4d8c541c897fd562fd3ad73c1 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,644 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.splits/4bb4acbdcedf2f8531e747ec59b78d24/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-4e1aa321a8e54e6a89b09587a5002040 for child: 4bb4acbdcedf2f8531e747ec59b78d24, parent: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,644 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.splits/307b832391c13214298b00417783c5e0/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-83e29b0da26a44bf9dee17b57dc4bfb0 for child: 307b832391c13214298b00417783c5e0, parent: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,644 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/83e29b0da26a44bf9dee17b57dc4bfb0 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,647 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/4e1aa321a8e54e6a89b09587a5002040 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,664 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.splits/4bb4acbdcedf2f8531e747ec59b78d24/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-f8f24e978d664e0bae7ec180d57833b9 for child: 4bb4acbdcedf2f8531e747ec59b78d24, parent: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,668 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/f8f24e978d664e0bae7ec180d57833b9 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,673 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.splits/307b832391c13214298b00417783c5e0/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-aa66485bbe564c6da61e1b86de6ab481 for child: 307b832391c13214298b00417783c5e0, parent: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,673 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/aa66485bbe564c6da61e1b86de6ab481 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,691 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 96848d837e2f45e098225129736df1e9 2023-05-04 10:54:15,691 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/2bc73271d81c4c30aa4473b1ed7ca1c5 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,692 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4e78c3f056a94e26b2e7a35b0572da21 2023-05-04 10:54:15,693 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/4d11ba96a1c4419ab7fc92485270da71 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,695 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 96848d837e2f45e098225129736df1e9 2023-05-04 10:54:15,695 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/96848d837e2f45e098225129736df1e9, top=true 2023-05-04 10:54:15,710 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a68f3a23f2b4f22a848d740c34ee4b7 2023-05-04 10:54:15,720 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/3728cf3691c143f3a2905a728159a750 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,720 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,732 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/.splits/307b832391c13214298b00417783c5e0/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-96848d837e2f45e098225129736df1e9 for child: 307b832391c13214298b00417783c5e0, parent: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,732 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/96848d837e2f45e098225129736df1e9 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,732 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/a2fba6e70f544183867c0b71a71f8730 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,750 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,751 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/6b50d2c64923457dacd08b63dbe0f201 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,752 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/4e78c3f056a94e26b2e7a35b0572da21 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,759 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/e9099b31bca44169b7db92f474f0015f for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,759 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/0a68f3a23f2b4f22a848d740c34ee4b7 for region: 6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:15,759 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 6e4db4ef42259e1e3f31375f39c7adca Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-04 10:54:15,812 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-05-04 10:54:15,814 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-05-04 10:54:15,817 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1683197655816"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1683197655816"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1683197655816"}]},"ts":"1683197655816"} 2023-05-04 10:54:15,817 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683197655816"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197655816"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197655816"}]},"ts":"1683197655816"} 2023-05-04 10:54:15,817 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683197655816"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197655816"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197655816"}]},"ts":"1683197655816"} 2023-05-04 10:54:15,821 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4bb4acbdcedf2f8531e747ec59b78d24, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=307b832391c13214298b00417783c5e0, ASSIGN}] 2023-05-04 10:54:15,822 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=307b832391c13214298b00417783c5e0, ASSIGN 2023-05-04 10:54:15,823 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=307b832391c13214298b00417783c5e0, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42285,1683197625528; forceNewPlan=false, retain=false 2023-05-04 10:54:15,824 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4bb4acbdcedf2f8531e747ec59b78d24, ASSIGN 2023-05-04 10:54:15,824 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4bb4acbdcedf2f8531e747ec59b78d24, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42285,1683197625528; forceNewPlan=false, retain=false 2023-05-04 10:54:15,973 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-04 10:54:15,975 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=4bb4acbdcedf2f8531e747ec59b78d24, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,975 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683197655975"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197655975"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197655975"}]},"ts":"1683197655975"} 2023-05-04 10:54:15,977 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure 4bb4acbdcedf2f8531e747ec59b78d24, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:54:15,979 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=307b832391c13214298b00417783c5e0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:15,979 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683197655979"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197655979"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197655979"}]},"ts":"1683197655979"} 2023-05-04 10:54:15,980 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure 307b832391c13214298b00417783c5e0, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:54:16,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. 2023-05-04 10:54:16,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 307b832391c13214298b00417783c5e0, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.', STARTKEY => '60', ENDKEY => ''} 2023-05-04 10:54:16,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:54:16,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 307b832391c13214298b00417783c5e0 2023-05-04 10:54:16,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:54:16,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 307b832391c13214298b00417783c5e0 2023-05-04 10:54:16,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 307b832391c13214298b00417783c5e0 2023-05-04 10:54:16,133 INFO [StoreOpener-307b832391c13214298b00417783c5e0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 307b832391c13214298b00417783c5e0 2023-05-04 10:54:16,134 DEBUG [StoreOpener-307b832391c13214298b00417783c5e0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1 2023-05-04 10:54:16,134 DEBUG [StoreOpener-307b832391c13214298b00417783c5e0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1 2023-05-04 10:54:16,135 INFO [StoreOpener-307b832391c13214298b00417783c5e0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 307b832391c13214298b00417783c5e0 columnFamilyName cf1 2023-05-04 10:54:16,146 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a68f3a23f2b4f22a848d740c34ee4b7.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:16,147 DEBUG [StoreOpener-307b832391c13214298b00417783c5e0-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/0a68f3a23f2b4f22a848d740c34ee4b7.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/0a68f3a23f2b4f22a848d740c34ee4b7-top 2023-05-04 10:54:16,151 DEBUG [StoreOpener-307b832391c13214298b00417783c5e0-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2-top 2023-05-04 10:54:16,155 DEBUG [StoreOpener-307b832391c13214298b00417783c5e0-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/6b50d2c64923457dacd08b63dbe0f201.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/6b50d2c64923457dacd08b63dbe0f201-top 2023-05-04 10:54:16,159 DEBUG [StoreOpener-307b832391c13214298b00417783c5e0-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/e9099b31bca44169b7db92f474f0015f.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/e9099b31bca44169b7db92f474f0015f-top 2023-05-04 10:54:16,162 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 83e29b0da26a44bf9dee17b57dc4bfb0 2023-05-04 10:54:16,162 DEBUG [StoreOpener-307b832391c13214298b00417783c5e0-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-83e29b0da26a44bf9dee17b57dc4bfb0 2023-05-04 10:54:16,165 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 96848d837e2f45e098225129736df1e9 2023-05-04 10:54:16,165 DEBUG [StoreOpener-307b832391c13214298b00417783c5e0-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-96848d837e2f45e098225129736df1e9 2023-05-04 10:54:16,168 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aa66485bbe564c6da61e1b86de6ab481 2023-05-04 10:54:16,168 DEBUG [StoreOpener-307b832391c13214298b00417783c5e0-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-aa66485bbe564c6da61e1b86de6ab481 2023-05-04 10:54:16,169 INFO [StoreOpener-307b832391c13214298b00417783c5e0-1] regionserver.HStore(310): Store=307b832391c13214298b00417783c5e0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:54:16,169 INFO [StoreOpener-307b832391c13214298b00417783c5e0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 307b832391c13214298b00417783c5e0 2023-05-04 10:54:16,170 DEBUG [StoreOpener-307b832391c13214298b00417783c5e0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2 2023-05-04 10:54:16,170 DEBUG [StoreOpener-307b832391c13214298b00417783c5e0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2 2023-05-04 10:54:16,170 INFO [StoreOpener-307b832391c13214298b00417783c5e0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 307b832391c13214298b00417783c5e0 columnFamilyName cf2 2023-05-04 10:54:16,183 DEBUG [StoreOpener-307b832391c13214298b00417783c5e0-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/3728cf3691c143f3a2905a728159a750.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/3728cf3691c143f3a2905a728159a750-top 2023-05-04 10:54:16,186 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4e78c3f056a94e26b2e7a35b0572da21.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:16,187 DEBUG [StoreOpener-307b832391c13214298b00417783c5e0-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/4e78c3f056a94e26b2e7a35b0572da21.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/4e78c3f056a94e26b2e7a35b0572da21-top 2023-05-04 10:54:16,192 DEBUG [StoreOpener-307b832391c13214298b00417783c5e0-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/a2fba6e70f544183867c0b71a71f8730.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/a2fba6e70f544183867c0b71a71f8730-top 2023-05-04 10:54:16,195 DEBUG [StoreOpener-307b832391c13214298b00417783c5e0-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf-top 2023-05-04 10:54:16,198 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 315c805320ed452280fc907b4c3b8204 2023-05-04 10:54:16,198 DEBUG [StoreOpener-307b832391c13214298b00417783c5e0-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-315c805320ed452280fc907b4c3b8204 2023-05-04 10:54:16,201 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 60f398e4d8c541c897fd562fd3ad73c1 2023-05-04 10:54:16,201 DEBUG [StoreOpener-307b832391c13214298b00417783c5e0-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-60f398e4d8c541c897fd562fd3ad73c1 2023-05-04 10:54:16,205 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 70eb3fe40ebf40349a58cd4c168f2344 2023-05-04 10:54:16,205 DEBUG [StoreOpener-307b832391c13214298b00417783c5e0-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-70eb3fe40ebf40349a58cd4c168f2344 2023-05-04 10:54:16,205 INFO [StoreOpener-307b832391c13214298b00417783c5e0-1] regionserver.HStore(310): Store=307b832391c13214298b00417783c5e0/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:54:16,206 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0 2023-05-04 10:54:16,207 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0 2023-05-04 10:54:16,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:54:16,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 307b832391c13214298b00417783c5e0 2023-05-04 10:54:16,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 307b832391c13214298b00417783c5e0; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71250723, jitterRate=0.06171850860118866}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:54:16,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 307b832391c13214298b00417783c5e0: 2023-05-04 10:54:16,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0., pid=78, masterSystemTime=1683197656128 2023-05-04 10:54:16,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:54:16,214 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:54:16,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:54:16,214 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:54:16,214 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. 2023-05-04 10:54:16,214 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. 2023-05-04 10:54:16,214 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1912): 307b832391c13214298b00417783c5e0/cf1 is initiating minor compaction (all files) 2023-05-04 10:54:16,214 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1912): 307b832391c13214298b00417783c5e0/cf2 is initiating minor compaction (all files) 2023-05-04 10:54:16,214 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 307b832391c13214298b00417783c5e0/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. 2023-05-04 10:54:16,214 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 307b832391c13214298b00417783c5e0/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. 2023-05-04 10:54:16,215 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/6b50d2c64923457dacd08b63dbe0f201.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/6b50d2c64923457dacd08b63dbe0f201-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/e9099b31bca44169b7db92f474f0015f.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/e9099b31bca44169b7db92f474f0015f-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/0a68f3a23f2b4f22a848d740c34ee4b7.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/0a68f3a23f2b4f22a848d740c34ee4b7-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-83e29b0da26a44bf9dee17b57dc4bfb0, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-aa66485bbe564c6da61e1b86de6ab481, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-96848d837e2f45e098225129736df1e9] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/.tmp, totalSize=34.8 K 2023-05-04 10:54:16,215 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/a2fba6e70f544183867c0b71a71f8730.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/a2fba6e70f544183867c0b71a71f8730-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/3728cf3691c143f3a2905a728159a750.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/3728cf3691c143f3a2905a728159a750-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/4e78c3f056a94e26b2e7a35b0572da21.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/4e78c3f056a94e26b2e7a35b0572da21-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-70eb3fe40ebf40349a58cd4c168f2344, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-60f398e4d8c541c897fd562fd3ad73c1, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-315c805320ed452280fc907b4c3b8204] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/.tmp, totalSize=34.8 K 2023-05-04 10:54:16,215 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting 6b50d2c64923457dacd08b63dbe0f201.6e4db4ef42259e1e3f31375f39c7adca, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683197653625 2023-05-04 10:54:16,215 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting a2fba6e70f544183867c0b71a71f8730.6e4db4ef42259e1e3f31375f39c7adca, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683197653625 2023-05-04 10:54:16,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. 2023-05-04 10:54:16,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. 2023-05-04 10:54:16,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. 2023-05-04 10:54:16,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4bb4acbdcedf2f8531e747ec59b78d24, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.', STARTKEY => '', ENDKEY => '60'} 2023-05-04 10:54:16,216 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=307b832391c13214298b00417783c5e0, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:16,216 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683197656216"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197656216"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197656216"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197656216"}]},"ts":"1683197656216"} 2023-05-04 10:54:16,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:54:16,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 4bb4acbdcedf2f8531e747ec59b78d24 2023-05-04 10:54:16,217 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting 42ae1011767f4eaf8e0fa0c6fb2d6ad2.6e4db4ef42259e1e3f31375f39c7adca, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683197653838 2023-05-04 10:54:16,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:54:16,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4bb4acbdcedf2f8531e747ec59b78d24 2023-05-04 10:54:16,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4bb4acbdcedf2f8531e747ec59b78d24 2023-05-04 10:54:16,218 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting e9099b31bca44169b7db92f474f0015f.6e4db4ef42259e1e3f31375f39c7adca, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683197654014 2023-05-04 10:54:16,219 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting 3728cf3691c143f3a2905a728159a750.6e4db4ef42259e1e3f31375f39c7adca, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683197653838 2023-05-04 10:54:16,219 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting 0a68f3a23f2b4f22a848d740c34ee4b7.6e4db4ef42259e1e3f31375f39c7adca, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1683197654758 2023-05-04 10:54:16,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-05-04 10:54:16,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure 307b832391c13214298b00417783c5e0, server=jenkins-hbase4.apache.org,42285,1683197625528 in 238 msec 2023-05-04 10:54:16,220 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting ed99ad8c0a1f4453b6bb3214812a6bcf.6e4db4ef42259e1e3f31375f39c7adca, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683197654014 2023-05-04 10:54:16,221 INFO [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4bb4acbdcedf2f8531e747ec59b78d24 2023-05-04 10:54:16,221 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting 4e78c3f056a94e26b2e7a35b0572da21.6e4db4ef42259e1e3f31375f39c7adca, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1683197654758 2023-05-04 10:54:16,221 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-83e29b0da26a44bf9dee17b57dc4bfb0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-05-04 10:54:16,221 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-70eb3fe40ebf40349a58cd4c168f2344, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-05-04 10:54:16,222 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-aa66485bbe564c6da61e1b86de6ab481, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-05-04 10:54:16,222 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=307b832391c13214298b00417783c5e0, ASSIGN in 399 msec 2023-05-04 10:54:16,222 DEBUG [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1 2023-05-04 10:54:16,222 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-96848d837e2f45e098225129736df1e9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-05-04 10:54:16,222 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-60f398e4d8c541c897fd562fd3ad73c1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-05-04 10:54:16,222 DEBUG [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1 2023-05-04 10:54:16,223 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-315c805320ed452280fc907b4c3b8204, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-05-04 10:54:16,223 INFO [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4bb4acbdcedf2f8531e747ec59b78d24 columnFamilyName cf1 2023-05-04 10:54:16,236 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a68f3a23f2b4f22a848d740c34ee4b7.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:16,236 DEBUG [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/0a68f3a23f2b4f22a848d740c34ee4b7.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/0a68f3a23f2b4f22a848d740c34ee4b7-bottom 2023-05-04 10:54:16,246 DEBUG [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2-bottom 2023-05-04 10:54:16,249 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] throttle.PressureAwareThroughputController(145): 307b832391c13214298b00417783c5e0#cf2#compaction#72 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:54:16,249 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] throttle.PressureAwareThroughputController(145): 307b832391c13214298b00417783c5e0#cf1#compaction#73 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:54:16,251 DEBUG [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/6b50d2c64923457dacd08b63dbe0f201.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/6b50d2c64923457dacd08b63dbe0f201-bottom 2023-05-04 10:54:16,255 DEBUG [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/e9099b31bca44169b7db92f474f0015f.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/e9099b31bca44169b7db92f474f0015f-bottom 2023-05-04 10:54:16,259 DEBUG [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-4d11ba96a1c4419ab7fc92485270da71 2023-05-04 10:54:16,263 DEBUG [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-4e1aa321a8e54e6a89b09587a5002040 2023-05-04 10:54:16,272 DEBUG [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-f8f24e978d664e0bae7ec180d57833b9 2023-05-04 10:54:16,272 INFO [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] regionserver.HStore(310): Store=4bb4acbdcedf2f8531e747ec59b78d24/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:54:16,272 INFO [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4bb4acbdcedf2f8531e747ec59b78d24 2023-05-04 10:54:16,273 DEBUG [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2 2023-05-04 10:54:16,273 DEBUG [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2 2023-05-04 10:54:16,274 INFO [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4bb4acbdcedf2f8531e747ec59b78d24 columnFamilyName cf2 2023-05-04 10:54:16,279 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for b172ed86d03e4f629af6cd90bb9efdd1: NONE, but ROW specified in column family configuration 2023-05-04 10:54:16,279 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/.tmp/cf2/b172ed86d03e4f629af6cd90bb9efdd1 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/b172ed86d03e4f629af6cd90bb9efdd1 2023-05-04 10:54:16,281 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 30f01c935a684ca2b4de1061e9015229: NONE, but ROW specified in column family configuration 2023-05-04 10:54:16,282 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/.tmp/cf1/30f01c935a684ca2b4de1061e9015229 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/30f01c935a684ca2b4de1061e9015229 2023-05-04 10:54:16,285 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for b172ed86d03e4f629af6cd90bb9efdd1: NONE, but ROW specified in column family configuration 2023-05-04 10:54:16,286 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 307b832391c13214298b00417783c5e0/cf2 of 307b832391c13214298b00417783c5e0 into b172ed86d03e4f629af6cd90bb9efdd1(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:54:16,286 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 307b832391c13214298b00417783c5e0: 2023-05-04 10:54:16,286 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0., storeName=307b832391c13214298b00417783c5e0/cf2, priority=9, startTime=1683197656214; duration=0sec 2023-05-04 10:54:16,286 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:54:16,287 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 30f01c935a684ca2b4de1061e9015229: NONE, but ROW specified in column family configuration 2023-05-04 10:54:16,288 DEBUG [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/3728cf3691c143f3a2905a728159a750.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/3728cf3691c143f3a2905a728159a750-bottom 2023-05-04 10:54:16,288 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 307b832391c13214298b00417783c5e0/cf1 of 307b832391c13214298b00417783c5e0 into 30f01c935a684ca2b4de1061e9015229(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:54:16,288 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 307b832391c13214298b00417783c5e0: 2023-05-04 10:54:16,288 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0., storeName=307b832391c13214298b00417783c5e0/cf1, priority=9, startTime=1683197656213; duration=0sec 2023-05-04 10:54:16,288 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:54:16,292 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4e78c3f056a94e26b2e7a35b0572da21.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:16,292 DEBUG [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/4e78c3f056a94e26b2e7a35b0572da21.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/4e78c3f056a94e26b2e7a35b0572da21-bottom 2023-05-04 10:54:16,296 DEBUG [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/a2fba6e70f544183867c0b71a71f8730.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/a2fba6e70f544183867c0b71a71f8730-bottom 2023-05-04 10:54:16,300 DEBUG [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf-bottom 2023-05-04 10:54:16,304 DEBUG [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-2bc73271d81c4c30aa4473b1ed7ca1c5 2023-05-04 10:54:16,307 DEBUG [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-314d84ca037f4e72a11fe4feda85e8e6 2023-05-04 10:54:16,316 DEBUG [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-65eaa2b193064baea11fcdb4a27b3a9d 2023-05-04 10:54:16,317 INFO [StoreOpener-4bb4acbdcedf2f8531e747ec59b78d24-1] regionserver.HStore(310): Store=4bb4acbdcedf2f8531e747ec59b78d24/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:54:16,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24 2023-05-04 10:54:16,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24 2023-05-04 10:54:16,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:54:16,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4bb4acbdcedf2f8531e747ec59b78d24 2023-05-04 10:54:16,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4bb4acbdcedf2f8531e747ec59b78d24; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62082788, jitterRate=-0.07489436864852905}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:54:16,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4bb4acbdcedf2f8531e747ec59b78d24: 2023-05-04 10:54:16,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24., pid=77, masterSystemTime=1683197656128 2023-05-04 10:54:16,322 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:54:16,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-04 10:54:16,323 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:54:16,323 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:54:16,323 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. 2023-05-04 10:54:16,323 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1912): 4bb4acbdcedf2f8531e747ec59b78d24/cf1 is initiating minor compaction (all files) 2023-05-04 10:54:16,323 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. 2023-05-04 10:54:16,323 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 4bb4acbdcedf2f8531e747ec59b78d24/cf1 in testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. 2023-05-04 10:54:16,323 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1912): 4bb4acbdcedf2f8531e747ec59b78d24/cf2 is initiating minor compaction (all files) 2023-05-04 10:54:16,323 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 4bb4acbdcedf2f8531e747ec59b78d24/cf2 in testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. 2023-05-04 10:54:16,324 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/6b50d2c64923457dacd08b63dbe0f201.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/6b50d2c64923457dacd08b63dbe0f201-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/e9099b31bca44169b7db92f474f0015f.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/e9099b31bca44169b7db92f474f0015f-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-4e1aa321a8e54e6a89b09587a5002040, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-f8f24e978d664e0bae7ec180d57833b9, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-4d11ba96a1c4419ab7fc92485270da71, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/0a68f3a23f2b4f22a848d740c34ee4b7.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/0a68f3a23f2b4f22a848d740c34ee4b7-bottom] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/.tmp, totalSize=35.0 K 2023-05-04 10:54:16,324 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/a2fba6e70f544183867c0b71a71f8730.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/a2fba6e70f544183867c0b71a71f8730-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/3728cf3691c143f3a2905a728159a750.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/3728cf3691c143f3a2905a728159a750-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-314d84ca037f4e72a11fe4feda85e8e6, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-65eaa2b193064baea11fcdb4a27b3a9d, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-2bc73271d81c4c30aa4473b1ed7ca1c5, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/4e78c3f056a94e26b2e7a35b0572da21.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/4e78c3f056a94e26b2e7a35b0572da21-bottom] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/.tmp, totalSize=35.0 K 2023-05-04 10:54:16,324 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting 6b50d2c64923457dacd08b63dbe0f201.6e4db4ef42259e1e3f31375f39c7adca, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683197653625 2023-05-04 10:54:16,324 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting a2fba6e70f544183867c0b71a71f8730.6e4db4ef42259e1e3f31375f39c7adca, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683197653625 2023-05-04 10:54:16,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. 2023-05-04 10:54:16,325 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting 42ae1011767f4eaf8e0fa0c6fb2d6ad2.6e4db4ef42259e1e3f31375f39c7adca, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683197653838 2023-05-04 10:54:16,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. 2023-05-04 10:54:16,325 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting e9099b31bca44169b7db92f474f0015f.6e4db4ef42259e1e3f31375f39c7adca, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683197654014 2023-05-04 10:54:16,325 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-4e1aa321a8e54e6a89b09587a5002040, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683197654192 2023-05-04 10:54:16,326 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-f8f24e978d664e0bae7ec180d57833b9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683197654381 2023-05-04 10:54:16,326 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-4d11ba96a1c4419ab7fc92485270da71, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683197654562 2023-05-04 10:54:16,326 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting 0a68f3a23f2b4f22a848d740c34ee4b7.6e4db4ef42259e1e3f31375f39c7adca, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683197654758 2023-05-04 10:54:16,328 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting 3728cf3691c143f3a2905a728159a750.6e4db4ef42259e1e3f31375f39c7adca, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683197653838 2023-05-04 10:54:16,328 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=4bb4acbdcedf2f8531e747ec59b78d24, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:16,328 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting ed99ad8c0a1f4453b6bb3214812a6bcf.6e4db4ef42259e1e3f31375f39c7adca, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683197654014 2023-05-04 10:54:16,328 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683197656328"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197656328"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197656328"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197656328"}]},"ts":"1683197656328"} 2023-05-04 10:54:16,329 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-314d84ca037f4e72a11fe4feda85e8e6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683197654192 2023-05-04 10:54:16,329 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-65eaa2b193064baea11fcdb4a27b3a9d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683197654381 2023-05-04 10:54:16,330 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-2bc73271d81c4c30aa4473b1ed7ca1c5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683197654562 2023-05-04 10:54:16,330 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting 4e78c3f056a94e26b2e7a35b0572da21.6e4db4ef42259e1e3f31375f39c7adca, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683197654758 2023-05-04 10:54:16,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-05-04 10:54:16,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure 4bb4acbdcedf2f8531e747ec59b78d24, server=jenkins-hbase4.apache.org,42285,1683197625528 in 354 msec 2023-05-04 10:54:16,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=75, resume processing ppid=72 2023-05-04 10:54:16,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4bb4acbdcedf2f8531e747ec59b78d24, ASSIGN in 512 msec 2023-05-04 10:54:16,337 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=6e4db4ef42259e1e3f31375f39c7adca, daughterA=4bb4acbdcedf2f8531e747ec59b78d24, daughterB=307b832391c13214298b00417783c5e0 in 1.0400 sec 2023-05-04 10:54:16,348 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4bb4acbdcedf2f8531e747ec59b78d24#cf1#compaction#74 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:54:16,356 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] throttle.PressureAwareThroughputController(145): 4bb4acbdcedf2f8531e747ec59b78d24#cf2#compaction#75 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:54:16,373 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/.tmp/cf1/31c99463225a48f0aba8f515bd41aec0 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/31c99463225a48f0aba8f515bd41aec0 2023-05-04 10:54:16,375 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/.tmp/cf2/df530a9f4b9f4d7cb6b08dd60e51da86 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/df530a9f4b9f4d7cb6b08dd60e51da86 2023-05-04 10:54:16,378 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4bb4acbdcedf2f8531e747ec59b78d24/cf1 of 4bb4acbdcedf2f8531e747ec59b78d24 into 31c99463225a48f0aba8f515bd41aec0(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:54:16,378 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 4bb4acbdcedf2f8531e747ec59b78d24: 2023-05-04 10:54:16,378 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24., storeName=4bb4acbdcedf2f8531e747ec59b78d24/cf1, priority=9, startTime=1683197656322; duration=0sec 2023-05-04 10:54:16,378 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:54:16,383 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4bb4acbdcedf2f8531e747ec59b78d24/cf2 of 4bb4acbdcedf2f8531e747ec59b78d24 into df530a9f4b9f4d7cb6b08dd60e51da86(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:54:16,383 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 4bb4acbdcedf2f8531e747ec59b78d24: 2023-05-04 10:54:16,383 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24., storeName=4bb4acbdcedf2f8531e747ec59b78d24/cf2, priority=9, startTime=1683197656323; duration=0sec 2023-05-04 10:54:16,383 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:54:16,557 DEBUG [Listener at localhost/35483] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-05-04 10:54:16,560 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33448, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-05-04 10:54:16,560 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42285] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. 2023-05-04 10:54:16,561 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42285] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-05-04 10:54:16,561 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42285] regionserver.HStore(1912): 4bb4acbdcedf2f8531e747ec59b78d24/cf1 is initiating major compaction (all files) 2023-05-04 10:54:16,561 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42285] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4c903a99; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:54:16,561 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 4bb4acbdcedf2f8531e747ec59b78d24/cf1 in testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. 2023-05-04 10:54:16,561 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42285] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-05-04 10:54:16,562 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/31c99463225a48f0aba8f515bd41aec0] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/.tmp, totalSize=6.8 K 2023-05-04 10:54:16,562 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42285] regionserver.HStore(1912): 4bb4acbdcedf2f8531e747ec59b78d24/cf2 is initiating major compaction (all files) 2023-05-04 10:54:16,562 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42285] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1663a415; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:54:16,562 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 4bb4acbdcedf2f8531e747ec59b78d24/cf2 in testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. 2023-05-04 10:54:16,562 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/df530a9f4b9f4d7cb6b08dd60e51da86] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/.tmp, totalSize=6.8 K 2023-05-04 10:54:16,562 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting 31c99463225a48f0aba8f515bd41aec0, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683197653625 2023-05-04 10:54:16,562 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting df530a9f4b9f4d7cb6b08dd60e51da86, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683197653625 2023-05-04 10:54:16,564 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42285] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. 2023-05-04 10:54:16,564 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42285] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-05-04 10:54:16,564 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42285] regionserver.HStore(1912): 307b832391c13214298b00417783c5e0/cf1 is initiating major compaction (all files) 2023-05-04 10:54:16,564 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42285] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@d8b46a7; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-04 10:54:16,564 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42285] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-05-04 10:54:16,564 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42285] regionserver.HStore(1912): 307b832391c13214298b00417783c5e0/cf2 is initiating major compaction (all files) 2023-05-04 10:54:16,564 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42285] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7209dd18; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-05-04 10:54:16,565 INFO [Listener at localhost/35483] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-05-04 10:54:16,567 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4bb4acbdcedf2f8531e747ec59b78d24#cf1#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:54:16,567 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] throttle.PressureAwareThroughputController(145): 4bb4acbdcedf2f8531e747ec59b78d24#cf2#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:54:16,591 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42285] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. is not mergeable because it has references 2023-05-04 10:54:16,592 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42285] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. is not mergeable because it has references 2023-05-04 10:54:16,596 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/.tmp/cf1/0d1f67ac6cf64841bfa0cabd4ebe3440 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/0d1f67ac6cf64841bfa0cabd4ebe3440 2023-05-04 10:54:16,601 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 4bb4acbdcedf2f8531e747ec59b78d24/cf1 of 4bb4acbdcedf2f8531e747ec59b78d24 into 0d1f67ac6cf64841bfa0cabd4ebe3440(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:54:16,601 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 4bb4acbdcedf2f8531e747ec59b78d24: 2023-05-04 10:54:16,602 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1683197656561; duration=0sec 2023-05-04 10:54:16,602 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-05-04 10:54:16,602 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 307b832391c13214298b00417783c5e0/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. 2023-05-04 10:54:16,603 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/30f01c935a684ca2b4de1061e9015229] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/.tmp, totalSize=5.0 K 2023-05-04 10:54:16,603 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting 30f01c935a684ca2b4de1061e9015229, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-05-04 10:54:16,606 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] throttle.PressureAwareThroughputController(145): 307b832391c13214298b00417783c5e0#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:54:16,620 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 7746a3a93c2d4f7e90203bf75b3a1b5d: NONE, but ROW specified in column family configuration 2023-05-04 10:54:16,620 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/.tmp/cf1/7746a3a93c2d4f7e90203bf75b3a1b5d as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/7746a3a93c2d4f7e90203bf75b3a1b5d 2023-05-04 10:54:16,624 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 7746a3a93c2d4f7e90203bf75b3a1b5d: NONE, but ROW specified in column family configuration 2023-05-04 10:54:16,625 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 307b832391c13214298b00417783c5e0/cf1 of 307b832391c13214298b00417783c5e0 into 7746a3a93c2d4f7e90203bf75b3a1b5d(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:54:16,625 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 307b832391c13214298b00417783c5e0: 2023-05-04 10:54:16,626 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1683197656564; duration=0sec 2023-05-04 10:54:16,626 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-04 10:54:16,626 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 307b832391c13214298b00417783c5e0/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. 2023-05-04 10:54:16,626 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/b172ed86d03e4f629af6cd90bb9efdd1] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/.tmp, totalSize=5.0 K 2023-05-04 10:54:16,626 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting b172ed86d03e4f629af6cd90bb9efdd1, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-05-04 10:54:16,629 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] throttle.PressureAwareThroughputController(145): 307b832391c13214298b00417783c5e0#cf2#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:54:16,640 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 0a640ea1a2d546eeb8e26bc07845f374: NONE, but ROW specified in column family configuration 2023-05-04 10:54:16,641 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/.tmp/cf2/0a640ea1a2d546eeb8e26bc07845f374 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/0a640ea1a2d546eeb8e26bc07845f374 2023-05-04 10:54:16,645 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 0a640ea1a2d546eeb8e26bc07845f374: NONE, but ROW specified in column family configuration 2023-05-04 10:54:16,646 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 307b832391c13214298b00417783c5e0/cf2 of 307b832391c13214298b00417783c5e0 into 0a640ea1a2d546eeb8e26bc07845f374(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:54:16,646 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 307b832391c13214298b00417783c5e0: 2023-05-04 10:54:16,646 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1683197656564; duration=0sec 2023-05-04 10:54:16,646 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:54:16,702 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42285] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. is not mergeable because it has references 2023-05-04 10:54:16,702 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42285] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. is not mergeable because it has references 2023-05-04 10:54:16,812 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42285] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. is not mergeable because it has references 2023-05-04 10:54:16,813 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42285] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. is not mergeable because it has references 2023-05-04 10:54:16,921 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42285] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. is not mergeable because it has references 2023-05-04 10:54:16,921 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42285] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. is not mergeable because it has references 2023-05-04 10:54:16,997 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/.tmp/cf2/89b06d7f6059437db396986509e6f569 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/89b06d7f6059437db396986509e6f569 2023-05-04 10:54:17,002 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 4bb4acbdcedf2f8531e747ec59b78d24/cf2 of 4bb4acbdcedf2f8531e747ec59b78d24 into 89b06d7f6059437db396986509e6f569(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:54:17,002 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 4bb4acbdcedf2f8531e747ec59b78d24: 2023-05-04 10:54:17,002 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1683197656562; duration=0sec 2023-05-04 10:54:17,002 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:54:17,016 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-04 10:54:17,030 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42285] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. is not mergeable because it has references 2023-05-04 10:54:17,031 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42285] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. is not mergeable because it has references 2023-05-04 10:54:17,037 DEBUG [Listener at localhost/35483] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-05-04 10:54:17,039 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:54:17,040 INFO [Listener at localhost/35483] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:17,041 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:17,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:17,045 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197657045"}]},"ts":"1683197657045"} 2023-05-04 10:54:17,046 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-05-04 10:54:17,048 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-05-04 10:54:17,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4bb4acbdcedf2f8531e747ec59b78d24, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=307b832391c13214298b00417783c5e0, UNASSIGN}] 2023-05-04 10:54:17,049 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=307b832391c13214298b00417783c5e0, UNASSIGN 2023-05-04 10:54:17,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-05-04 10:54:17,050 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=307b832391c13214298b00417783c5e0, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:17,050 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683197657050"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197657050"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197657050"}]},"ts":"1683197657050"} 2023-05-04 10:54:17,051 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 307b832391c13214298b00417783c5e0, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:54:17,054 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4bb4acbdcedf2f8531e747ec59b78d24, UNASSIGN 2023-05-04 10:54:17,054 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=4bb4acbdcedf2f8531e747ec59b78d24, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:17,055 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683197657054"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197657054"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197657054"}]},"ts":"1683197657054"} 2023-05-04 10:54:17,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 4bb4acbdcedf2f8531e747ec59b78d24, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:54:17,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-05-04 10:54:17,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 4bb4acbdcedf2f8531e747ec59b78d24 2023-05-04 10:54:17,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4bb4acbdcedf2f8531e747ec59b78d24, disabling compactions & flushes 2023-05-04 10:54:17,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. 2023-05-04 10:54:17,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. 2023-05-04 10:54:17,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. after waiting 0 ms 2023-05-04 10:54:17,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. 2023-05-04 10:54:17,204 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/6b50d2c64923457dacd08b63dbe0f201.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/6b50d2c64923457dacd08b63dbe0f201-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/e9099b31bca44169b7db92f474f0015f.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/e9099b31bca44169b7db92f474f0015f-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-4e1aa321a8e54e6a89b09587a5002040, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-f8f24e978d664e0bae7ec180d57833b9, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-4d11ba96a1c4419ab7fc92485270da71, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/31c99463225a48f0aba8f515bd41aec0, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/0a68f3a23f2b4f22a848d740c34ee4b7.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/0a68f3a23f2b4f22a848d740c34ee4b7-bottom] to archive 2023-05-04 10:54:17,205 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:54:17,207 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/6b50d2c64923457dacd08b63dbe0f201.6e4db4ef42259e1e3f31375f39c7adca to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/6b50d2c64923457dacd08b63dbe0f201.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:17,208 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2.6e4db4ef42259e1e3f31375f39c7adca to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:17,209 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/e9099b31bca44169b7db92f474f0015f.6e4db4ef42259e1e3f31375f39c7adca to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/e9099b31bca44169b7db92f474f0015f.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:17,210 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-4e1aa321a8e54e6a89b09587a5002040 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-4e1aa321a8e54e6a89b09587a5002040 2023-05-04 10:54:17,211 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-f8f24e978d664e0bae7ec180d57833b9 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-f8f24e978d664e0bae7ec180d57833b9 2023-05-04 10:54:17,212 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-4d11ba96a1c4419ab7fc92485270da71 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-4d11ba96a1c4419ab7fc92485270da71 2023-05-04 10:54:17,214 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/31c99463225a48f0aba8f515bd41aec0 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/31c99463225a48f0aba8f515bd41aec0 2023-05-04 10:54:17,215 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/0a68f3a23f2b4f22a848d740c34ee4b7.6e4db4ef42259e1e3f31375f39c7adca to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/0a68f3a23f2b4f22a848d740c34ee4b7.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:17,216 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/a2fba6e70f544183867c0b71a71f8730.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/a2fba6e70f544183867c0b71a71f8730-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/3728cf3691c143f3a2905a728159a750.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/3728cf3691c143f3a2905a728159a750-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-314d84ca037f4e72a11fe4feda85e8e6, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-65eaa2b193064baea11fcdb4a27b3a9d, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-2bc73271d81c4c30aa4473b1ed7ca1c5, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/df530a9f4b9f4d7cb6b08dd60e51da86, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/4e78c3f056a94e26b2e7a35b0572da21.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/4e78c3f056a94e26b2e7a35b0572da21-bottom] to archive 2023-05-04 10:54:17,217 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:54:17,218 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/a2fba6e70f544183867c0b71a71f8730.6e4db4ef42259e1e3f31375f39c7adca to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/a2fba6e70f544183867c0b71a71f8730.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:17,219 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/3728cf3691c143f3a2905a728159a750.6e4db4ef42259e1e3f31375f39c7adca to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/3728cf3691c143f3a2905a728159a750.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:17,220 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf.6e4db4ef42259e1e3f31375f39c7adca to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:17,221 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-314d84ca037f4e72a11fe4feda85e8e6 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-314d84ca037f4e72a11fe4feda85e8e6 2023-05-04 10:54:17,222 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-65eaa2b193064baea11fcdb4a27b3a9d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-65eaa2b193064baea11fcdb4a27b3a9d 2023-05-04 10:54:17,223 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-2bc73271d81c4c30aa4473b1ed7ca1c5 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-2bc73271d81c4c30aa4473b1ed7ca1c5 2023-05-04 10:54:17,224 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/df530a9f4b9f4d7cb6b08dd60e51da86 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/df530a9f4b9f4d7cb6b08dd60e51da86 2023-05-04 10:54:17,225 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/4e78c3f056a94e26b2e7a35b0572da21.6e4db4ef42259e1e3f31375f39c7adca to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/4e78c3f056a94e26b2e7a35b0572da21.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:17,229 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-05-04 10:54:17,229 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:54:17,230 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24. 2023-05-04 10:54:17,230 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4bb4acbdcedf2f8531e747ec59b78d24: 2023-05-04 10:54:17,232 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 4bb4acbdcedf2f8531e747ec59b78d24 2023-05-04 10:54:17,232 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 307b832391c13214298b00417783c5e0 2023-05-04 10:54:17,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 307b832391c13214298b00417783c5e0, disabling compactions & flushes 2023-05-04 10:54:17,232 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. 2023-05-04 10:54:17,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. 2023-05-04 10:54:17,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. after waiting 0 ms 2023-05-04 10:54:17,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. 2023-05-04 10:54:17,232 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=4bb4acbdcedf2f8531e747ec59b78d24, regionState=CLOSED 2023-05-04 10:54:17,233 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683197657232"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197657232"}]},"ts":"1683197657232"} 2023-05-04 10:54:17,234 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/6b50d2c64923457dacd08b63dbe0f201.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/6b50d2c64923457dacd08b63dbe0f201-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/e9099b31bca44169b7db92f474f0015f.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/e9099b31bca44169b7db92f474f0015f-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/0a68f3a23f2b4f22a848d740c34ee4b7.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/0a68f3a23f2b4f22a848d740c34ee4b7-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-83e29b0da26a44bf9dee17b57dc4bfb0, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-aa66485bbe564c6da61e1b86de6ab481, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/30f01c935a684ca2b4de1061e9015229, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-96848d837e2f45e098225129736df1e9] to archive 2023-05-04 10:54:17,234 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:54:17,236 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-05-04 10:54:17,236 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 4bb4acbdcedf2f8531e747ec59b78d24, server=jenkins-hbase4.apache.org,42285,1683197625528 in 178 msec 2023-05-04 10:54:17,237 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/6b50d2c64923457dacd08b63dbe0f201.6e4db4ef42259e1e3f31375f39c7adca to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/6b50d2c64923457dacd08b63dbe0f201.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:17,237 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4bb4acbdcedf2f8531e747ec59b78d24, UNASSIGN in 188 msec 2023-05-04 10:54:17,238 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2.6e4db4ef42259e1e3f31375f39c7adca to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:17,239 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/e9099b31bca44169b7db92f474f0015f.6e4db4ef42259e1e3f31375f39c7adca to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/e9099b31bca44169b7db92f474f0015f.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:17,240 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/0a68f3a23f2b4f22a848d740c34ee4b7.6e4db4ef42259e1e3f31375f39c7adca to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/0a68f3a23f2b4f22a848d740c34ee4b7.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:17,241 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-83e29b0da26a44bf9dee17b57dc4bfb0 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-83e29b0da26a44bf9dee17b57dc4bfb0 2023-05-04 10:54:17,242 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-aa66485bbe564c6da61e1b86de6ab481 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-aa66485bbe564c6da61e1b86de6ab481 2023-05-04 10:54:17,243 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/30f01c935a684ca2b4de1061e9015229 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/30f01c935a684ca2b4de1061e9015229 2023-05-04 10:54:17,244 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-96848d837e2f45e098225129736df1e9 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-96848d837e2f45e098225129736df1e9 2023-05-04 10:54:17,245 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/a2fba6e70f544183867c0b71a71f8730.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/a2fba6e70f544183867c0b71a71f8730-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/3728cf3691c143f3a2905a728159a750.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/3728cf3691c143f3a2905a728159a750-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/4e78c3f056a94e26b2e7a35b0572da21.6e4db4ef42259e1e3f31375f39c7adca->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/4e78c3f056a94e26b2e7a35b0572da21-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-70eb3fe40ebf40349a58cd4c168f2344, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-60f398e4d8c541c897fd562fd3ad73c1, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/b172ed86d03e4f629af6cd90bb9efdd1, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-315c805320ed452280fc907b4c3b8204] to archive 2023-05-04 10:54:17,246 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:54:17,247 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/a2fba6e70f544183867c0b71a71f8730.6e4db4ef42259e1e3f31375f39c7adca to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/a2fba6e70f544183867c0b71a71f8730.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:17,249 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/3728cf3691c143f3a2905a728159a750.6e4db4ef42259e1e3f31375f39c7adca to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/3728cf3691c143f3a2905a728159a750.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:17,250 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf.6e4db4ef42259e1e3f31375f39c7adca to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:17,251 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/4e78c3f056a94e26b2e7a35b0572da21.6e4db4ef42259e1e3f31375f39c7adca to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/4e78c3f056a94e26b2e7a35b0572da21.6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:17,252 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-70eb3fe40ebf40349a58cd4c168f2344 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-70eb3fe40ebf40349a58cd4c168f2344 2023-05-04 10:54:17,253 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-60f398e4d8c541c897fd562fd3ad73c1 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-60f398e4d8c541c897fd562fd3ad73c1 2023-05-04 10:54:17,254 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/b172ed86d03e4f629af6cd90bb9efdd1 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/b172ed86d03e4f629af6cd90bb9efdd1 2023-05-04 10:54:17,255 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-315c805320ed452280fc907b4c3b8204 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/testSplitTableRegionDeletedRowsDaughter=6e4db4ef42259e1e3f31375f39c7adca-315c805320ed452280fc907b4c3b8204 2023-05-04 10:54:17,259 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-05-04 10:54:17,259 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:54:17,260 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0. 2023-05-04 10:54:17,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 307b832391c13214298b00417783c5e0: 2023-05-04 10:54:17,262 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 307b832391c13214298b00417783c5e0 2023-05-04 10:54:17,262 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=307b832391c13214298b00417783c5e0, regionState=CLOSED 2023-05-04 10:54:17,262 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683197657262"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197657262"}]},"ts":"1683197657262"} 2023-05-04 10:54:17,265 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-05-04 10:54:17,265 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 307b832391c13214298b00417783c5e0, server=jenkins-hbase4.apache.org,42285,1683197625528 in 213 msec 2023-05-04 10:54:17,266 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-05-04 10:54:17,267 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=307b832391c13214298b00417783c5e0, UNASSIGN in 217 msec 2023-05-04 10:54:17,267 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197657267"}]},"ts":"1683197657267"} 2023-05-04 10:54:17,269 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-05-04 10:54:17,270 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-05-04 10:54:17,272 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 230 msec 2023-05-04 10:54:17,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-05-04 10:54:17,353 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-05-04 10:54:17,353 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:17,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:17,356 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:17,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-05-04 10:54:17,356 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:17,361 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:17,361 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0 2023-05-04 10:54:17,361 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24 2023-05-04 10:54:17,363 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/recovered.edits] 2023-05-04 10:54:17,363 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/recovered.edits] 2023-05-04 10:54:17,363 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/recovered.edits] 2023-05-04 10:54:17,366 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/7746a3a93c2d4f7e90203bf75b3a1b5d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf1/7746a3a93c2d4f7e90203bf75b3a1b5d 2023-05-04 10:54:17,367 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/0d1f67ac6cf64841bfa0cabd4ebe3440 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf1/0d1f67ac6cf64841bfa0cabd4ebe3440 2023-05-04 10:54:17,367 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/0a68f3a23f2b4f22a848d740c34ee4b7 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/0a68f3a23f2b4f22a848d740c34ee4b7 2023-05-04 10:54:17,368 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/42ae1011767f4eaf8e0fa0c6fb2d6ad2 2023-05-04 10:54:17,369 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/0a640ea1a2d546eeb8e26bc07845f374 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/cf2/0a640ea1a2d546eeb8e26bc07845f374 2023-05-04 10:54:17,369 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/89b06d7f6059437db396986509e6f569 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/cf2/89b06d7f6059437db396986509e6f569 2023-05-04 10:54:17,369 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/4d11ba96a1c4419ab7fc92485270da71 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/4d11ba96a1c4419ab7fc92485270da71 2023-05-04 10:54:17,371 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/4e1aa321a8e54e6a89b09587a5002040 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/4e1aa321a8e54e6a89b09587a5002040 2023-05-04 10:54:17,372 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/6b50d2c64923457dacd08b63dbe0f201 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/6b50d2c64923457dacd08b63dbe0f201 2023-05-04 10:54:17,372 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/recovered.edits/113.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24/recovered.edits/113.seqid 2023-05-04 10:54:17,372 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/recovered.edits/113.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0/recovered.edits/113.seqid 2023-05-04 10:54:17,373 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4bb4acbdcedf2f8531e747ec59b78d24 2023-05-04 10:54:17,373 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/307b832391c13214298b00417783c5e0 2023-05-04 10:54:17,374 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/83e29b0da26a44bf9dee17b57dc4bfb0 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/83e29b0da26a44bf9dee17b57dc4bfb0 2023-05-04 10:54:17,375 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/96848d837e2f45e098225129736df1e9 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/96848d837e2f45e098225129736df1e9 2023-05-04 10:54:17,376 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/aa66485bbe564c6da61e1b86de6ab481 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/aa66485bbe564c6da61e1b86de6ab481 2023-05-04 10:54:17,377 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/e9099b31bca44169b7db92f474f0015f to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/e9099b31bca44169b7db92f474f0015f 2023-05-04 10:54:17,378 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/f8f24e978d664e0bae7ec180d57833b9 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf1/f8f24e978d664e0bae7ec180d57833b9 2023-05-04 10:54:17,380 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/2bc73271d81c4c30aa4473b1ed7ca1c5 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/2bc73271d81c4c30aa4473b1ed7ca1c5 2023-05-04 10:54:17,381 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/314d84ca037f4e72a11fe4feda85e8e6 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/314d84ca037f4e72a11fe4feda85e8e6 2023-05-04 10:54:17,382 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/315c805320ed452280fc907b4c3b8204 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/315c805320ed452280fc907b4c3b8204 2023-05-04 10:54:17,383 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/3728cf3691c143f3a2905a728159a750 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/3728cf3691c143f3a2905a728159a750 2023-05-04 10:54:17,384 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/4e78c3f056a94e26b2e7a35b0572da21 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/4e78c3f056a94e26b2e7a35b0572da21 2023-05-04 10:54:17,385 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/60f398e4d8c541c897fd562fd3ad73c1 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/60f398e4d8c541c897fd562fd3ad73c1 2023-05-04 10:54:17,386 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/65eaa2b193064baea11fcdb4a27b3a9d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/65eaa2b193064baea11fcdb4a27b3a9d 2023-05-04 10:54:17,387 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/70eb3fe40ebf40349a58cd4c168f2344 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/70eb3fe40ebf40349a58cd4c168f2344 2023-05-04 10:54:17,388 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/a2fba6e70f544183867c0b71a71f8730 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/a2fba6e70f544183867c0b71a71f8730 2023-05-04 10:54:17,389 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/cf2/ed99ad8c0a1f4453b6bb3214812a6bcf 2023-05-04 10:54:17,391 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/recovered.edits/106.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca/recovered.edits/106.seqid 2023-05-04 10:54:17,391 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6e4db4ef42259e1e3f31375f39c7adca 2023-05-04 10:54:17,391 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-05-04 10:54:17,393 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:17,396 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-05-04 10:54:17,397 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-05-04 10:54:17,398 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:17,398 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-05-04 10:54:17,399 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197657399"}]},"ts":"9223372036854775807"} 2023-05-04 10:54:17,399 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197657399"}]},"ts":"9223372036854775807"} 2023-05-04 10:54:17,399 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197657399"}]},"ts":"9223372036854775807"} 2023-05-04 10:54:17,400 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-04 10:54:17,401 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 6e4db4ef42259e1e3f31375f39c7adca, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1683197653121.6e4db4ef42259e1e3f31375f39c7adca.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 4bb4acbdcedf2f8531e747ec59b78d24, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1683197655295.4bb4acbdcedf2f8531e747ec59b78d24.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 307b832391c13214298b00417783c5e0, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1683197655295.307b832391c13214298b00417783c5e0.', STARTKEY => '60', ENDKEY => ''}] 2023-05-04 10:54:17,401 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-05-04 10:54:17,401 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683197657401"}]},"ts":"9223372036854775807"} 2023-05-04 10:54:17,402 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-05-04 10:54:17,404 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:54:17,405 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 51 msec 2023-05-04 10:54:17,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-05-04 10:54:17,458 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-05-04 10:54:17,472 INFO [Listener at localhost/35483] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=414 (was 407) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:36606 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:52146 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:40210 [Waiting for operation #32] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:39950 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:58124 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:53362 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:39714 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:39780 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:58428 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:51834 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:45026 [Waiting for operation #36] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:40202 [Waiting for operation #33] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:58212 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:40004 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2d8012cd-shared-pool-18 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:52240 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2137743924_15 at /127.0.0.1:33304 [Waiting for operation #45] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=862 (was 840) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=349 (was 319) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=6415 (was 6439) 2023-05-04 10:54:17,486 INFO [Listener at localhost/35483] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=414, OpenFileDescriptor=862, MaxFileDescriptor=60000, SystemLoadAverage=349, ProcessCount=170, AvailableMemoryMB=6414 2023-05-04 10:54:17,487 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:54:17,489 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-04 10:54:17,490 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-04 10:54:17,491 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:54:17,493 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-05-04 10:54:17,493 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-04 10:54:17,494 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:54:17,495 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc empty. 2023-05-04 10:54:17,495 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:54:17,495 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-05-04 10:54:17,508 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-05-04 10:54:17,509 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7675): creating {ENCODED => 9de9ca68de03518c48ec4415c14457cc, NAME => 'testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp 2023-05-04 10:54:17,516 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:54:17,516 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1604): Closing 9de9ca68de03518c48ec4415c14457cc, disabling compactions & flushes 2023-05-04 10:54:17,516 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. 2023-05-04 10:54:17,516 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. 2023-05-04 10:54:17,516 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. after waiting 0 ms 2023-05-04 10:54:17,516 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. 2023-05-04 10:54:17,516 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. 2023-05-04 10:54:17,516 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1558): Region close journal for 9de9ca68de03518c48ec4415c14457cc: 2023-05-04 10:54:17,518 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-05-04 10:54:17,519 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683197657519"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197657519"}]},"ts":"1683197657519"} 2023-05-04 10:54:17,521 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-04 10:54:17,521 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-04 10:54:17,522 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197657522"}]},"ts":"1683197657522"} 2023-05-04 10:54:17,522 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-05-04 10:54:17,525 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-04 10:54:17,526 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-04 10:54:17,526 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-04 10:54:17,526 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-04 10:54:17,526 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-04 10:54:17,526 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=9de9ca68de03518c48ec4415c14457cc, ASSIGN}] 2023-05-04 10:54:17,527 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=9de9ca68de03518c48ec4415c14457cc, ASSIGN 2023-05-04 10:54:17,528 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=9de9ca68de03518c48ec4415c14457cc, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33581,1683197625489; forceNewPlan=false, retain=false 2023-05-04 10:54:17,678 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-04 10:54:17,681 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=9de9ca68de03518c48ec4415c14457cc, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:17,681 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683197657681"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197657681"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197657681"}]},"ts":"1683197657681"} 2023-05-04 10:54:17,682 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 9de9ca68de03518c48ec4415c14457cc, server=jenkins-hbase4.apache.org,33581,1683197625489}] 2023-05-04 10:54:17,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. 2023-05-04 10:54:17,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9de9ca68de03518c48ec4415c14457cc, NAME => 'testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.', STARTKEY => '', ENDKEY => ''} 2023-05-04 10:54:17,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:54:17,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:54:17,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:54:17,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:54:17,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:54:17,840 INFO [StoreOpener-9de9ca68de03518c48ec4415c14457cc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:54:17,842 DEBUG [StoreOpener-9de9ca68de03518c48ec4415c14457cc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc/cf1 2023-05-04 10:54:17,842 DEBUG [StoreOpener-9de9ca68de03518c48ec4415c14457cc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc/cf1 2023-05-04 10:54:17,842 INFO [StoreOpener-9de9ca68de03518c48ec4415c14457cc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9de9ca68de03518c48ec4415c14457cc columnFamilyName cf1 2023-05-04 10:54:17,843 INFO [StoreOpener-9de9ca68de03518c48ec4415c14457cc-1] regionserver.HStore(310): Store=9de9ca68de03518c48ec4415c14457cc/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:54:17,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:54:17,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:54:17,847 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:54:17,849 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-04 10:54:17,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 9de9ca68de03518c48ec4415c14457cc; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72043825, jitterRate=0.07353664934635162}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-04 10:54:17,851 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 9de9ca68de03518c48ec4415c14457cc: 2023-05-04 10:54:17,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc., pid=87, masterSystemTime=1683197657834 2023-05-04 10:54:17,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. 2023-05-04 10:54:17,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. 2023-05-04 10:54:17,854 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=9de9ca68de03518c48ec4415c14457cc, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:17,854 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683197657854"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197657854"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197657854"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197657854"}]},"ts":"1683197657854"} 2023-05-04 10:54:17,857 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-05-04 10:54:17,857 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 9de9ca68de03518c48ec4415c14457cc, server=jenkins-hbase4.apache.org,33581,1683197625489 in 173 msec 2023-05-04 10:54:17,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-05-04 10:54:17,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=9de9ca68de03518c48ec4415c14457cc, ASSIGN in 331 msec 2023-05-04 10:54:17,860 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-04 10:54:17,860 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197657860"}]},"ts":"1683197657860"} 2023-05-04 10:54:17,862 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-05-04 10:54:17,867 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-05-04 10:54:17,868 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 377 msec 2023-05-04 10:54:18,003 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-05-04 10:54:18,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:54:18,022 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-05-04 10:54:18,023 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-05-04 10:54:18,023 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-05-04 10:54:18,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-05-04 10:54:18,039 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=9de9ca68de03518c48ec4415c14457cc, REOPEN/MOVE}] 2023-05-04 10:54:18,040 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=9de9ca68de03518c48ec4415c14457cc, REOPEN/MOVE 2023-05-04 10:54:18,041 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=9de9ca68de03518c48ec4415c14457cc, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:18,041 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683197658041"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197658041"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197658041"}]},"ts":"1683197658041"} 2023-05-04 10:54:18,042 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 9de9ca68de03518c48ec4415c14457cc, server=jenkins-hbase4.apache.org,33581,1683197625489}] 2023-05-04 10:54:18,194 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:54:18,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 9de9ca68de03518c48ec4415c14457cc, disabling compactions & flushes 2023-05-04 10:54:18,195 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. 2023-05-04 10:54:18,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. 2023-05-04 10:54:18,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. after waiting 0 ms 2023-05-04 10:54:18,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. 2023-05-04 10:54:18,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-05-04 10:54:18,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:54:18,199 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. 2023-05-04 10:54:18,199 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 9de9ca68de03518c48ec4415c14457cc: 2023-05-04 10:54:18,199 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 9de9ca68de03518c48ec4415c14457cc to self. 2023-05-04 10:54:18,200 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:54:18,201 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=9de9ca68de03518c48ec4415c14457cc, regionState=CLOSED 2023-05-04 10:54:18,201 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683197658201"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197658201"}]},"ts":"1683197658201"} 2023-05-04 10:54:18,203 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-05-04 10:54:18,203 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 9de9ca68de03518c48ec4415c14457cc, server=jenkins-hbase4.apache.org,33581,1683197625489 in 160 msec 2023-05-04 10:54:18,204 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=9de9ca68de03518c48ec4415c14457cc, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,33581,1683197625489; forceNewPlan=false, retain=true 2023-05-04 10:54:18,354 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-04 10:54:18,354 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=9de9ca68de03518c48ec4415c14457cc, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:18,355 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683197658354"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197658354"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197658354"}]},"ts":"1683197658354"} 2023-05-04 10:54:18,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 9de9ca68de03518c48ec4415c14457cc, server=jenkins-hbase4.apache.org,33581,1683197625489}] 2023-05-04 10:54:18,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. 2023-05-04 10:54:18,511 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9de9ca68de03518c48ec4415c14457cc, NAME => 'testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.', STARTKEY => '', ENDKEY => ''} 2023-05-04 10:54:18,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:54:18,511 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:54:18,511 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:54:18,511 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:54:18,511 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:54:18,512 INFO [StoreOpener-9de9ca68de03518c48ec4415c14457cc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:54:18,513 DEBUG [StoreOpener-9de9ca68de03518c48ec4415c14457cc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc/cf1 2023-05-04 10:54:18,513 DEBUG [StoreOpener-9de9ca68de03518c48ec4415c14457cc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc/cf1 2023-05-04 10:54:18,513 INFO [StoreOpener-9de9ca68de03518c48ec4415c14457cc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9de9ca68de03518c48ec4415c14457cc columnFamilyName cf1 2023-05-04 10:54:18,514 INFO [StoreOpener-9de9ca68de03518c48ec4415c14457cc-1] regionserver.HStore(310): Store=9de9ca68de03518c48ec4415c14457cc/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:54:18,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:54:18,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:54:18,518 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:54:18,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 9de9ca68de03518c48ec4415c14457cc; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73932131, jitterRate=0.10167460143566132}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-04 10:54:18,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 9de9ca68de03518c48ec4415c14457cc: 2023-05-04 10:54:18,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc., pid=92, masterSystemTime=1683197658508 2023-05-04 10:54:18,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. 2023-05-04 10:54:18,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. 2023-05-04 10:54:18,521 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=9de9ca68de03518c48ec4415c14457cc, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:54:18,522 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683197658521"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197658521"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197658521"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197658521"}]},"ts":"1683197658521"} 2023-05-04 10:54:18,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-05-04 10:54:18,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 9de9ca68de03518c48ec4415c14457cc, server=jenkins-hbase4.apache.org,33581,1683197625489 in 167 msec 2023-05-04 10:54:18,526 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-05-04 10:54:18,526 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=9de9ca68de03518c48ec4415c14457cc, REOPEN/MOVE in 485 msec 2023-05-04 10:54:18,529 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-05-04 10:54:18,529 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 501 msec 2023-05-04 10:54:18,532 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98, ASSIGN}] 2023-05-04 10:54:18,533 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98, ASSIGN 2023-05-04 10:54:18,534 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-05-04 10:54:18,684 DEBUG [jenkins-hbase4:40233] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-04 10:54:18,685 DEBUG [jenkins-hbase4:40233] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-04 10:54:18,685 DEBUG [jenkins-hbase4:40233] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-04 10:54:18,685 DEBUG [jenkins-hbase4:40233] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-04 10:54:18,685 DEBUG [jenkins-hbase4:40233] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-05-04 10:54:18,687 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=afb93548d078e4e75c09efee8b4edf98, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:54:18,687 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683197658686"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1683197658686"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1683197658686"}]},"ts":"1683197658686"} 2023-05-04 10:54:18,690 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure afb93548d078e4e75c09efee8b4edf98, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:54:18,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1683197657490_0001.afb93548d078e4e75c09efee8b4edf98. 2023-05-04 10:54:18,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => afb93548d078e4e75c09efee8b4edf98, NAME => 'testRollbackForSplitTableRegionWithReplica,,1683197657490_0001.afb93548d078e4e75c09efee8b4edf98.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-05-04 10:54:18,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:54:18,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica afb93548d078e4e75c09efee8b4edf98 2023-05-04 10:54:18,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1683197657490_0001.afb93548d078e4e75c09efee8b4edf98.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:54:18,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for afb93548d078e4e75c09efee8b4edf98 2023-05-04 10:54:18,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for afb93548d078e4e75c09efee8b4edf98 2023-05-04 10:54:18,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:18,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:19,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:19,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:19,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:19,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:19,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:19,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:19,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:19,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:19,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:19,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:20,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:20,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:20,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:20,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:20,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:20,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:20,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:20,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:20,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:20,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:21,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:21,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:21,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:21,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:21,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:21,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:21,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:21,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:21,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:21,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:22,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:22,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:22,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:22,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:22,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:22,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:22,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:22,731 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-04 10:54:22,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:22,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:22,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:23,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:23,066 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-05-04 10:54:23,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:23,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:23,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:23,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:23,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:23,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:23,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:23,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:23,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:24,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:24,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:24,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:24,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:24,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:24,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:24,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:24,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:24,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:24,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:25,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:25,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:25,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:25,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:25,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:25,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:25,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:25,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:25,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:25,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:26,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:26,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:26,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:26,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:26,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:26,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:26,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:26,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:26,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:26,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:27,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:27,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:27,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:27,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:27,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:27,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:27,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:27,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:27,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:27,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:28,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:28,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:28,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:28,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:28,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:28,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:28,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:28,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:28,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:28,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:29,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:29,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:29,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:29,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:29,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:29,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:29,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:29,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:29,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:29,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:30,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:30,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:30,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:30,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:30,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:30,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:30,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:30,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:30,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:30,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:31,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:31,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:31,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:31,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:31,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:31,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:31,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:31,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:31,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:31,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:32,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:32,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:32,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:32,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:32,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:32,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:32,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:32,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:32,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:32,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:33,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:33,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:33,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:33,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:33,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:33,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:33,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:33,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:33,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:33,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:34,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:34,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:34,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:34,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:34,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:34,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:34,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:34,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:34,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:34,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:35,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:35,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:35,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:35,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:35,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:35,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:35,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:35,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:35,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:35,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:36,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:36,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:36,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:36,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:36,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:36,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:36,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:36,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:36,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:36,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:37,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:37,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:37,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:37,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:37,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:37,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:37,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:37,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:37,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:37,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:38,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:38,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:38,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:38,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:38,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:38,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:38,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:38,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:38,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:38,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:39,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:39,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:39,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:39,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:39,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:39,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:39,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:39,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:39,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:39,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:40,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:40,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:40,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:40,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:40,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:40,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:40,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:40,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:40,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:40,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:41,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:41,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:41,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:41,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:41,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:41,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:41,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:41,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:41,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:41,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:42,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:42,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:42,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:42,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:42,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:42,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:42,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:42,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:42,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:42,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:43,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:43,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:43,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:43,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:43,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:43,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:43,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:43,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:43,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:43,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:44,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:44,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:44,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:44,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:44,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:44,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:44,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:44,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:44,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:45,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:45,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:45,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:45,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:45,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:45,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:45,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:45,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:45,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:45,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:46,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:46,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:46,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:46,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:46,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:46,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:46,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:46,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:46,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:46,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:47,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:47,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:47,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:47,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:47,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:47,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:47,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:47,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:47,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:47,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:48,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:48,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:48,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:48,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:48,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:48,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:48,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:48,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:48,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:48,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:49,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:49,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:49,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:49,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:49,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:49,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:49,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:49,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:49,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:49,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:50,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:50,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:50,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:50,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:50,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:50,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:50,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:50,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:50,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:50,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:51,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:51,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:51,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:51,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:51,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:51,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:51,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:51,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:51,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:51,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:52,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:52,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:52,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:52,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:52,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:52,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:52,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:52,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:52,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:52,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:53,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:53,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:53,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:53,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:53,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:53,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:53,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:53,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:53,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:53,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:54,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:54,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:54,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:54,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:54,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:54,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:54,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:54,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:54,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:54,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:55,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:55,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:55,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:55,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:55,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:55,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:55,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:55,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:55,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:55,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:56,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:56,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:56,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:56,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:56,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:56,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:56,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:56,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:56,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:56,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:57,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:57,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:57,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:57,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:57,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:57,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:57,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:57,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:57,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:57,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:58,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:58,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:58,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:58,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:58,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:58,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:58,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:58,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:58,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:58,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:59,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:59,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:59,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:59,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:59,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:59,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:59,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:59,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:59,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:54:59,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:00,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:00,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:00,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:00,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:00,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:00,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:00,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:00,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:00,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:00,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:01,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:01,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:01,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:01,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:01,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:01,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:01,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:01,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:01,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:01,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:02,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:02,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:02,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:02,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:02,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:02,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:02,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:02,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:02,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:02,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:03,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:03,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:03,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:03,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:03,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:03,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:03,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:03,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:03,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:03,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:04,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:04,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:04,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:04,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:04,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:04,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:04,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:04,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:04,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:04,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:05,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:05,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:05,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:05,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:05,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:05,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:05,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:05,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:05,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:05,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:06,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:06,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:06,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:06,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:06,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:06,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:06,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:06,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:06,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:06,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:07,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:07,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:07,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:07,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:07,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:07,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:07,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:07,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:07,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:07,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:08,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:08,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:08,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:08,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:08,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:08,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:08,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:08,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:08,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:08,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:09,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:09,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:09,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:09,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:09,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:09,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:09,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:09,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:09,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:09,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:10,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:10,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:10,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:10,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:10,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:10,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:10,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:10,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:10,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:10,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:11,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:11,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:11,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:11,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:11,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:11,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:11,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:11,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:11,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:11,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:12,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:12,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:12,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:12,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:12,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:12,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:12,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:12,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:12,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:12,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:13,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:13,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:13,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:13,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:13,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:13,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:13,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:13,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:13,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:13,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:14,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:14,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:14,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:14,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:14,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:14,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:14,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:14,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:14,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:14,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:15,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:15,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:15,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:15,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:15,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:15,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:15,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:15,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:15,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:15,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:16,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:16,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:16,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:16,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:16,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:16,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:16,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:16,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:16,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:16,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:17,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:17,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:17,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:17,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:17,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:17,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:17,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:17,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:17,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:17,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:18,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:18,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-04 10:55:18,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:18,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-04 10:55:18,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:18,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:18,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-04 10:55:18,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:18,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:18,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:18,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-04 10:55:18,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:18,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:18,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:19,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:19,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:19,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-04 10:55:19,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:19,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:19,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:19,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:19,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:19,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:19,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:19,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:20,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:20,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:20,228 DEBUG [Listener at localhost/35483] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:20,229 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=9de9ca68de03518c48ec4415c14457cc, daughterA=248805fe1fdf27869faf29c5509ba1c6, daughterB=11f1194eaf50bbc8eb140bdfb097c7b7 2023-05-04 10:55:20,230 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=9de9ca68de03518c48ec4415c14457cc, daughterA=248805fe1fdf27869faf29c5509ba1c6, daughterB=11f1194eaf50bbc8eb140bdfb097c7b7 2023-05-04 10:55:20,230 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=9de9ca68de03518c48ec4415c14457cc, daughterA=248805fe1fdf27869faf29c5509ba1c6, daughterB=11f1194eaf50bbc8eb140bdfb097c7b7 2023-05-04 10:55:20,230 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=9de9ca68de03518c48ec4415c14457cc, daughterA=248805fe1fdf27869faf29c5509ba1c6, daughterB=11f1194eaf50bbc8eb140bdfb097c7b7 2023-05-04 10:55:20,232 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 9de9ca68de03518c48ec4415c14457cc, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=9de9ca68de03518c48ec4415c14457cc, daughterA=248805fe1fdf27869faf29c5509ba1c6, daughterB=11f1194eaf50bbc8eb140bdfb097c7b7 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42285,1683197625528, table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:55:20,235 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42285,1683197625528, table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=9de9ca68de03518c48ec4415c14457cc, daughterA=248805fe1fdf27869faf29c5509ba1c6, daughterB=11f1194eaf50bbc8eb140bdfb097c7b7 exec-time=6 msec 2023-05-04 10:55:20,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:20,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:20,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-04 10:55:20,580 INFO [StoreOpener-afb93548d078e4e75c09efee8b4edf98-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region afb93548d078e4e75c09efee8b4edf98 2023-05-04 10:55:20,583 DEBUG [StoreOpener-afb93548d078e4e75c09efee8b4edf98-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc/cf1 2023-05-04 10:55:20,583 DEBUG [StoreOpener-afb93548d078e4e75c09efee8b4edf98-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc/cf1 2023-05-04 10:55:20,583 INFO [StoreOpener-afb93548d078e4e75c09efee8b4edf98-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region afb93548d078e4e75c09efee8b4edf98 columnFamilyName cf1 2023-05-04 10:55:20,584 INFO [StoreOpener-afb93548d078e4e75c09efee8b4edf98-1] regionserver.HStore(310): Store=afb93548d078e4e75c09efee8b4edf98/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:20,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened afb93548d078e4e75c09efee8b4edf98; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72085542, jitterRate=0.07415828108787537}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-04 10:55:20,586 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for afb93548d078e4e75c09efee8b4edf98: 2023-05-04 10:55:20,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1683197657490_0001.afb93548d078e4e75c09efee8b4edf98., pid=94, masterSystemTime=1683197658842 2023-05-04 10:55:20,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1683197657490_0001.afb93548d078e4e75c09efee8b4edf98. 2023-05-04 10:55:20,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1683197657490_0001.afb93548d078e4e75c09efee8b4edf98. 2023-05-04 10:55:20,589 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=afb93548d078e4e75c09efee8b4edf98, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:20,589 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683197720588"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1683197720588"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1683197720588"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1683197720588"}]},"ts":"1683197720588"} 2023-05-04 10:55:20,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-05-04 10:55:20,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure afb93548d078e4e75c09efee8b4edf98, server=jenkins-hbase4.apache.org,42285,1683197625528 in 1 mins, 1.902 sec 2023-05-04 10:55:20,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-05-04 10:55:20,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98, ASSIGN in 1 mins, 2.061 sec 2023-05-04 10:55:20,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.588 sec 2023-05-04 10:55:20,595 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1683197658000","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62595,"client":"172.31.14.131:59962","queuetimems":0,"class":"HMaster"} 2023-05-04 10:55:20,596 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 631, skipping over this response of 3 bytes 2023-05-04 10:55:22,480 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42285,1683197625528, table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98 is currently in transition, give up 2023-05-04 10:55:22,482 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:55:22,483 INFO [Listener at localhost/35483] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:22,484 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:22,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:22,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-05-04 10:55:22,487 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197722487"}]},"ts":"1683197722487"} 2023-05-04 10:55:22,488 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-05-04 10:55:22,490 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-05-04 10:55:22,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=9de9ca68de03518c48ec4415c14457cc, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98, UNASSIGN}] 2023-05-04 10:55:22,491 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98, UNASSIGN 2023-05-04 10:55:22,492 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=afb93548d078e4e75c09efee8b4edf98, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:22,492 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683197722492"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1683197722492"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1683197722492"}]},"ts":"1683197722492"} 2023-05-04 10:55:22,494 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure afb93548d078e4e75c09efee8b4edf98, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:55:22,496 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=9de9ca68de03518c48ec4415c14457cc, UNASSIGN 2023-05-04 10:55:22,497 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=9de9ca68de03518c48ec4415c14457cc, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:22,497 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683197722497"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197722497"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197722497"}]},"ts":"1683197722497"} 2023-05-04 10:55:22,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 9de9ca68de03518c48ec4415c14457cc, server=jenkins-hbase4.apache.org,33581,1683197625489}] 2023-05-04 10:55:22,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-05-04 10:55:22,646 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close afb93548d078e4e75c09efee8b4edf98 2023-05-04 10:55:22,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing afb93548d078e4e75c09efee8b4edf98, disabling compactions & flushes 2023-05-04 10:55:22,647 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1683197657490_0001.afb93548d078e4e75c09efee8b4edf98. 2023-05-04 10:55:22,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1683197657490_0001.afb93548d078e4e75c09efee8b4edf98. 2023-05-04 10:55:22,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1683197657490_0001.afb93548d078e4e75c09efee8b4edf98. after waiting 0 ms 2023-05-04 10:55:22,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1683197657490_0001.afb93548d078e4e75c09efee8b4edf98. 2023-05-04 10:55:22,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:55:22,648 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1683197657490_0001.afb93548d078e4e75c09efee8b4edf98. 2023-05-04 10:55:22,648 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for afb93548d078e4e75c09efee8b4edf98: 2023-05-04 10:55:22,650 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed afb93548d078e4e75c09efee8b4edf98 2023-05-04 10:55:22,650 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:55:22,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 9de9ca68de03518c48ec4415c14457cc, disabling compactions & flushes 2023-05-04 10:55:22,651 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. 2023-05-04 10:55:22,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. 2023-05-04 10:55:22,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. after waiting 0 ms 2023-05-04 10:55:22,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. 2023-05-04 10:55:22,651 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=afb93548d078e4e75c09efee8b4edf98, regionState=CLOSED 2023-05-04 10:55:22,651 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683197722651"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1683197722651"}]},"ts":"1683197722651"} 2023-05-04 10:55:22,655 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-05-04 10:55:22,655 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure afb93548d078e4e75c09efee8b4edf98, server=jenkins-hbase4.apache.org,42285,1683197625528 in 160 msec 2023-05-04 10:55:22,655 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-05-04 10:55:22,656 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98, UNASSIGN in 164 msec 2023-05-04 10:55:22,656 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:55:22,657 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc. 2023-05-04 10:55:22,657 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 9de9ca68de03518c48ec4415c14457cc: 2023-05-04 10:55:22,658 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:55:22,658 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=9de9ca68de03518c48ec4415c14457cc, regionState=CLOSED 2023-05-04 10:55:22,658 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683197722658"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197722658"}]},"ts":"1683197722658"} 2023-05-04 10:55:22,661 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-05-04 10:55:22,661 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 9de9ca68de03518c48ec4415c14457cc, server=jenkins-hbase4.apache.org,33581,1683197625489 in 161 msec 2023-05-04 10:55:22,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-05-04 10:55:22,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=9de9ca68de03518c48ec4415c14457cc, UNASSIGN in 170 msec 2023-05-04 10:55:22,663 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197722663"}]},"ts":"1683197722663"} 2023-05-04 10:55:22,664 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-05-04 10:55:22,665 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-05-04 10:55:22,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 181 msec 2023-05-04 10:55:22,789 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-05-04 10:55:22,789 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-05-04 10:55:22,790 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:22,791 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:22,793 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:22,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-05-04 10:55:22,798 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:22,803 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:55:22,804 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc/recovered.edits] 2023-05-04 10:55:22,813 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc/recovered.edits/7.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc/recovered.edits/7.seqid 2023-05-04 10:55:22,813 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/9de9ca68de03518c48ec4415c14457cc 2023-05-04 10:55:22,814 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-05-04 10:55:22,816 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:22,818 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-05-04 10:55:22,820 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-05-04 10:55:22,821 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:22,821 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-05-04 10:55:22,821 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197722821"}]},"ts":"9223372036854775807"} 2023-05-04 10:55:22,821 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1683197657490_0001.afb93548d078e4e75c09efee8b4edf98.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197722821"}]},"ts":"9223372036854775807"} 2023-05-04 10:55:22,825 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-05-04 10:55:22,825 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 9de9ca68de03518c48ec4415c14457cc, NAME => 'testRollbackForSplitTableRegionWithReplica,,1683197657490.9de9ca68de03518c48ec4415c14457cc.', STARTKEY => '', ENDKEY => ''}, {ENCODED => afb93548d078e4e75c09efee8b4edf98, NAME => 'testRollbackForSplitTableRegionWithReplica,,1683197657490_0001.afb93548d078e4e75c09efee8b4edf98.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-05-04 10:55:22,825 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-05-04 10:55:22,825 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683197722825"}]},"ts":"9223372036854775807"} 2023-05-04 10:55:22,827 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-05-04 10:55:22,828 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:22,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 38 msec 2023-05-04 10:55:22,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-05-04 10:55:22,895 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-05-04 10:55:22,910 INFO [Listener at localhost/35483] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=347 (was 414), OpenFileDescriptor=655 (was 862), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=189 (was 349), ProcessCount=170 (was 170), AvailableMemoryMB=6380 (was 6414) 2023-05-04 10:55:22,924 INFO [Listener at localhost/35483] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=348, OpenFileDescriptor=655, MaxFileDescriptor=60000, SystemLoadAverage=189, ProcessCount=170, AvailableMemoryMB=6380 2023-05-04 10:55:22,925 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:55:22,927 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-04 10:55:22,928 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-04 10:55:22,929 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:22,931 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-05-04 10:55:22,932 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-04 10:55:22,933 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a 2023-05-04 10:55:22,934 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a empty. 2023-05-04 10:55:22,934 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a 2023-05-04 10:55:22,934 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-05-04 10:55:22,944 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-05-04 10:55:22,946 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7675): creating {ENCODED => bb802aa4052fe7e449f1c59066b7ab3a, NAME => 'testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp 2023-05-04 10:55:22,953 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:55:22,953 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1604): Closing bb802aa4052fe7e449f1c59066b7ab3a, disabling compactions & flushes 2023-05-04 10:55:22,953 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:22,953 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:22,953 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. after waiting 0 ms 2023-05-04 10:55:22,953 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:22,953 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:22,953 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1558): Region close journal for bb802aa4052fe7e449f1c59066b7ab3a: 2023-05-04 10:55:22,955 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-05-04 10:55:22,956 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1683197722956"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197722956"}]},"ts":"1683197722956"} 2023-05-04 10:55:22,957 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-04 10:55:22,958 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-04 10:55:22,958 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197722958"}]},"ts":"1683197722958"} 2023-05-04 10:55:22,959 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-05-04 10:55:22,962 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-04 10:55:22,963 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-04 10:55:22,963 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-04 10:55:22,963 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-04 10:55:22,963 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-04 10:55:22,963 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=bb802aa4052fe7e449f1c59066b7ab3a, ASSIGN}] 2023-05-04 10:55:22,964 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=bb802aa4052fe7e449f1c59066b7ab3a, ASSIGN 2023-05-04 10:55:22,965 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=bb802aa4052fe7e449f1c59066b7ab3a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33581,1683197625489; forceNewPlan=false, retain=false 2023-05-04 10:55:23,115 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-04 10:55:23,116 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=bb802aa4052fe7e449f1c59066b7ab3a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,117 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1683197723116"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197723116"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197723116"}]},"ts":"1683197723116"} 2023-05-04 10:55:23,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure bb802aa4052fe7e449f1c59066b7ab3a, server=jenkins-hbase4.apache.org,33581,1683197625489}] 2023-05-04 10:55:23,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:23,273 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bb802aa4052fe7e449f1c59066b7ab3a, NAME => 'testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a.', STARTKEY => '', ENDKEY => ''} 2023-05-04 10:55:23,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:55:23,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey bb802aa4052fe7e449f1c59066b7ab3a 2023-05-04 10:55:23,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:55:23,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bb802aa4052fe7e449f1c59066b7ab3a 2023-05-04 10:55:23,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bb802aa4052fe7e449f1c59066b7ab3a 2023-05-04 10:55:23,275 INFO [StoreOpener-bb802aa4052fe7e449f1c59066b7ab3a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region bb802aa4052fe7e449f1c59066b7ab3a 2023-05-04 10:55:23,276 DEBUG [StoreOpener-bb802aa4052fe7e449f1c59066b7ab3a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1 2023-05-04 10:55:23,277 DEBUG [StoreOpener-bb802aa4052fe7e449f1c59066b7ab3a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1 2023-05-04 10:55:23,277 INFO [StoreOpener-bb802aa4052fe7e449f1c59066b7ab3a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bb802aa4052fe7e449f1c59066b7ab3a columnFamilyName cf1 2023-05-04 10:55:23,278 INFO [StoreOpener-bb802aa4052fe7e449f1c59066b7ab3a-1] regionserver.HStore(310): Store=bb802aa4052fe7e449f1c59066b7ab3a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:23,278 INFO [StoreOpener-bb802aa4052fe7e449f1c59066b7ab3a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region bb802aa4052fe7e449f1c59066b7ab3a 2023-05-04 10:55:23,279 DEBUG [StoreOpener-bb802aa4052fe7e449f1c59066b7ab3a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2 2023-05-04 10:55:23,279 DEBUG [StoreOpener-bb802aa4052fe7e449f1c59066b7ab3a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2 2023-05-04 10:55:23,280 INFO [StoreOpener-bb802aa4052fe7e449f1c59066b7ab3a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bb802aa4052fe7e449f1c59066b7ab3a columnFamilyName cf2 2023-05-04 10:55:23,281 INFO [StoreOpener-bb802aa4052fe7e449f1c59066b7ab3a-1] regionserver.HStore(310): Store=bb802aa4052fe7e449f1c59066b7ab3a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:23,282 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a 2023-05-04 10:55:23,282 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a 2023-05-04 10:55:23,284 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:55:23,285 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for bb802aa4052fe7e449f1c59066b7ab3a 2023-05-04 10:55:23,288 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-04 10:55:23,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bb802aa4052fe7e449f1c59066b7ab3a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63796934, jitterRate=-0.04935160279273987}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:55:23,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bb802aa4052fe7e449f1c59066b7ab3a: 2023-05-04 10:55:23,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a., pid=104, masterSystemTime=1683197723270 2023-05-04 10:55:23,291 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:23,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:23,292 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=bb802aa4052fe7e449f1c59066b7ab3a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,292 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1683197723292"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197723292"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197723292"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197723292"}]},"ts":"1683197723292"} 2023-05-04 10:55:23,296 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-05-04 10:55:23,296 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure bb802aa4052fe7e449f1c59066b7ab3a, server=jenkins-hbase4.apache.org,33581,1683197625489 in 176 msec 2023-05-04 10:55:23,298 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-05-04 10:55:23,298 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=bb802aa4052fe7e449f1c59066b7ab3a, ASSIGN in 333 msec 2023-05-04 10:55:23,299 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-04 10:55:23,299 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197723299"}]},"ts":"1683197723299"} 2023-05-04 10:55:23,300 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-05-04 10:55:23,302 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-05-04 10:55:23,303 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 374 msec 2023-05-04 10:55:23,466 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:23,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-04 10:55:23,468 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-04 10:55:23,468 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:23,468 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-04 10:55:23,468 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-04 10:55:23,468 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,468 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,470 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,470 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:23,470 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:23,470 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:23,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:23,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:23,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:23,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:23,470 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:23,471 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:23,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:23,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:23,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:23,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,471 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-04 10:55:23,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:23,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,472 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-04 10:55:23,472 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-04 10:55:23,472 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-04 10:55:23,472 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-04 10:55:23,472 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-04 10:55:23,472 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-04 10:55:23,472 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:23,472 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-04 10:55:23,472 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:23,472 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-04 10:55:23,472 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-04 10:55:23,472 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-04 10:55:23,472 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-04 10:55:23,472 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:23,472 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-04 10:55:23,472 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-04 10:55:23,472 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-04 10:55:23,472 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-04 10:55:23,472 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:23,472 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:23,473 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:23,473 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. started... 2023-05-04 10:55:23,474 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bb802aa4052fe7e449f1c59066b7ab3a 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-04 10:55:23,474 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,475 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,475 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,475 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:23,475 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,475 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:23,486 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf1/bd80b0760ee94762a162655715963a02 2023-05-04 10:55:23,501 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf2/2bd92ed91bad4ef1a754484d24521173 2023-05-04 10:55:23,505 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf1/bd80b0760ee94762a162655715963a02 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/bd80b0760ee94762a162655715963a02 2023-05-04 10:55:23,509 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/bd80b0760ee94762a162655715963a02, entries=2, sequenceid=6, filesize=4.8 K 2023-05-04 10:55:23,510 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf2/2bd92ed91bad4ef1a754484d24521173 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/2bd92ed91bad4ef1a754484d24521173 2023-05-04 10:55:23,514 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/2bd92ed91bad4ef1a754484d24521173, entries=2, sequenceid=6, filesize=4.8 K 2023-05-04 10:55:23,515 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for bb802aa4052fe7e449f1c59066b7ab3a in 41ms, sequenceid=6, compaction requested=false 2023-05-04 10:55:23,515 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-05-04 10:55:23,515 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bb802aa4052fe7e449f1c59066b7ab3a: 2023-05-04 10:55:23,515 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:23,515 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:23,515 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:23,516 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,516 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-04 10:55:23,516 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-04 10:55:23,517 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,517 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:23,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:23,517 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,517 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:23,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:23,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:23,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:23,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:23,519 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-04 10:55:23,519 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c3c661[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:23,519 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-04 10:55:23,519 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,521 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,521 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,521 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,522 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-04 10:55:23,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,522 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,522 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:23,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,522 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-04 10:55:23,522 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-04 10:55:23,522 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-04 10:55:23,522 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-04 10:55:23,522 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:23,522 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-04 10:55:23,522 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:23,522 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:23,523 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-04 10:55:23,523 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-04 10:55:23,524 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:23,523 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:23,524 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-04 10:55:23,524 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-04 10:55:23,524 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,524 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-04 10:55:23,525 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:23,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:23,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:23,525 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-04 10:55:23,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:23,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:23,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:23,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:23,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:23,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33581,1683197625489': 2023-05-04 10:55:23,527 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-04 10:55:23,527 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-04 10:55:23,527 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:23,528 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:23,528 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-04 10:55:23,528 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:23,529 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,529 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,529 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:23,529 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,529 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:23,529 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,529 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,529 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:23,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:23,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:23,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:23,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:23,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:23,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:23,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:23,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:23,530 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:23,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,530 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:23,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:23,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:23,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,532 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,532 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:23,532 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:23,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,536 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:23,536 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:23,536 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:23,536 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:23,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:23,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:23,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:23,536 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:23,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:23,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:23,536 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:23,536 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:23,536 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-04 10:55:23,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:23,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:23,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:23,536 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:23,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:23,537 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:23,537 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:23,537 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:23,537 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,537 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,537 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,537 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,537 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,537 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:23,537 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,537 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:23,537 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,537 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,638 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:23,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:23,650 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:23,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:23,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-04 10:55:23,651 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-04 10:55:23,651 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:23,652 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-04 10:55:23,652 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-04 10:55:23,652 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,652 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,653 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:23,653 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:23,653 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:23,653 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:23,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:23,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:23,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:23,653 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,653 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:23,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:23,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:23,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,654 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,654 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,654 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:23,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,654 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-04 10:55:23,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:23,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:23,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,655 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-04 10:55:23,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,655 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-04 10:55:23,655 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-04 10:55:23,655 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-04 10:55:23,655 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:23,655 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-04 10:55:23,655 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-04 10:55:23,655 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-04 10:55:23,655 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-04 10:55:23,655 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-04 10:55:23,655 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:23,655 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:23,655 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:23,655 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-04 10:55:23,655 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-04 10:55:23,655 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:23,655 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-04 10:55:23,655 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. started... 2023-05-04 10:55:23,655 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-04 10:55:23,655 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-04 10:55:23,655 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bb802aa4052fe7e449f1c59066b7ab3a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:23,655 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-04 10:55:23,656 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:23,656 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-04 10:55:23,656 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-04 10:55:23,656 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-04 10:55:23,657 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,661 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,661 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,661 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:23,661 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,661 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:23,670 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf1/cd571975264247f2b6e28230994f891b 2023-05-04 10:55:23,686 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf2/ac94f4093c1b4ee3bed720a959838d80 2023-05-04 10:55:23,691 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf1/cd571975264247f2b6e28230994f891b as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/cd571975264247f2b6e28230994f891b 2023-05-04 10:55:23,694 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/cd571975264247f2b6e28230994f891b, entries=10, sequenceid=19, filesize=5.0 K 2023-05-04 10:55:23,695 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf2/ac94f4093c1b4ee3bed720a959838d80 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/ac94f4093c1b4ee3bed720a959838d80 2023-05-04 10:55:23,700 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/ac94f4093c1b4ee3bed720a959838d80, entries=10, sequenceid=19, filesize=5.0 K 2023-05-04 10:55:23,701 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for bb802aa4052fe7e449f1c59066b7ab3a in 46ms, sequenceid=19, compaction requested=false 2023-05-04 10:55:23,702 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bb802aa4052fe7e449f1c59066b7ab3a: 2023-05-04 10:55:23,702 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:23,702 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:23,702 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:23,702 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,702 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-04 10:55:23,702 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-04 10:55:23,703 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,703 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:23,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:23,704 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,704 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:23,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:23,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:23,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:23,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:23,706 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-04 10:55:23,706 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15aa0d01[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:23,706 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-04 10:55:23,706 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,707 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,707 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,707 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,707 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-04 10:55:23,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,707 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-04 10:55:23,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,708 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:23,708 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-04 10:55:23,708 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-04 10:55:23,708 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,708 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:23,708 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-04 10:55:23,708 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:23,708 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-04 10:55:23,708 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:23,709 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-04 10:55:23,709 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:23,709 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-04 10:55:23,711 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-04 10:55:23,711 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:23,711 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-04 10:55:23,711 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,711 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:23,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,711 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-04 10:55:23,711 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-04 10:55:23,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:23,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:23,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:23,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:23,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:23,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:23,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:23,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33581,1683197625489': 2023-05-04 10:55:23,715 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-04 10:55:23,715 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-04 10:55:23,715 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:23,715 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:23,715 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-04 10:55:23,715 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:23,717 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,717 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,717 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,717 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,717 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:23,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,717 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:23,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:23,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:23,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,717 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:23,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:23,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:23,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:23,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:23,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:23,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:23,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:23,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:23,718 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:23,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,718 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:23,719 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:23,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:23,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,721 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,721 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,721 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,723 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,724 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:23,724 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:23,724 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:23,724 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:23,724 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:23,724 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:23,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:23,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:23,724 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-04 10:55:23,724 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:23,724 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:23,724 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:23,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:23,724 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,724 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,724 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,724 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,724 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:23,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:23,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:23,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:23,724 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,725 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,725 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,725 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:23,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:23,724 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:23,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:23,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:23,726 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:23,826 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:23,827 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:23,840 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:23,841 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:23,841 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-04 10:55:23,841 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-04 10:55:23,841 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:23,841 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-04 10:55:23,842 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-04 10:55:23,842 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,842 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,843 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:23,843 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:23,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:23,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:23,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:23,843 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:23,843 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:23,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:23,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:23,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,844 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,844 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:23,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,844 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,844 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:23,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,844 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,844 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-04 10:55:23,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:23,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:23,845 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-04 10:55:23,845 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-04 10:55:23,845 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-04 10:55:23,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,845 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-04 10:55:23,845 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:23,845 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-04 10:55:23,845 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-04 10:55:23,845 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-04 10:55:23,845 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-04 10:55:23,845 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-04 10:55:23,845 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-04 10:55:23,845 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-04 10:55:23,845 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-04 10:55:23,845 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:23,845 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-04 10:55:23,845 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-04 10:55:23,845 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-04 10:55:23,845 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:23,845 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-04 10:55:23,845 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-04 10:55:23,845 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-04 10:55:23,845 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:23,846 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:23,846 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:23,846 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. started... 2023-05-04 10:55:23,846 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bb802aa4052fe7e449f1c59066b7ab3a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:23,847 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,847 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,847 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:23,848 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,848 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,849 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:23,855 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf1/d9f843e079fc41b8b97b52dbd3458512 2023-05-04 10:55:23,870 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf2/a8b9b0641a7e445daf5fb493a236b60e 2023-05-04 10:55:23,874 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf1/d9f843e079fc41b8b97b52dbd3458512 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/d9f843e079fc41b8b97b52dbd3458512 2023-05-04 10:55:23,878 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/d9f843e079fc41b8b97b52dbd3458512, entries=10, sequenceid=32, filesize=5.0 K 2023-05-04 10:55:23,879 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf2/a8b9b0641a7e445daf5fb493a236b60e as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/a8b9b0641a7e445daf5fb493a236b60e 2023-05-04 10:55:23,883 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/a8b9b0641a7e445daf5fb493a236b60e, entries=10, sequenceid=32, filesize=5.0 K 2023-05-04 10:55:23,884 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for bb802aa4052fe7e449f1c59066b7ab3a in 38ms, sequenceid=32, compaction requested=true 2023-05-04 10:55:23,884 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bb802aa4052fe7e449f1c59066b7ab3a: 2023-05-04 10:55:23,884 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:23,884 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:23,884 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:23,884 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,884 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-04 10:55:23,884 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-04 10:55:23,886 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,886 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:23,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:23,886 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,886 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:23,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:23,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:23,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:23,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:23,889 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-04 10:55:23,889 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@349d5f1e[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:23,889 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-04 10:55:23,889 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,890 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,890 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,890 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,890 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,890 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:23,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,890 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-04 10:55:23,890 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-04 10:55:23,890 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-04 10:55:23,890 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:23,890 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-04 10:55:23,891 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-04 10:55:23,890 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-04 10:55:23,891 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:23,891 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:23,893 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-04 10:55:23,893 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:23,893 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-04 10:55:23,894 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-04 10:55:23,894 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,894 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-04 10:55:23,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:23,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:23,894 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:23,894 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:23,894 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-04 10:55:23,894 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-04 10:55:23,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:23,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:23,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:23,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:23,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:23,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33581,1683197625489': 2023-05-04 10:55:23,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-04 10:55:23,897 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-04 10:55:23,897 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:23,897 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:23,897 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-04 10:55:23,897 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:23,899 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,899 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,899 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:23,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,899 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,899 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,899 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:23,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:23,899 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:23,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,899 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:23,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:23,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:23,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:23,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:23,900 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:23,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:23,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:23,900 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:23,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:23,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:23,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,902 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,902 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:23,902 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:23,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,908 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,908 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:23,908 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:23,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:23,908 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:23,908 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-04 10:55:23,908 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:23,908 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:23,908 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:23,908 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,908 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:23,908 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:23,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:23,908 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:23,908 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:23,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:23,909 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:23,908 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:23,909 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:23,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:23,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:23,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:23,909 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:23,909 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,909 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:23,909 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:23,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:23,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:23,909 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:23,909 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:23,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:23,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,009 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:24,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:24,022 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:24,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:24,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-04 10:55:24,023 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-04 10:55:24,023 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:24,023 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-04 10:55:24,023 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-04 10:55:24,023 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,024 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,025 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,025 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:24,025 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:24,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:24,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:24,025 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:24,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:24,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:24,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:24,025 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,025 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:24,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:24,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,026 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,026 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,026 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:24,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:24,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:24,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,026 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-04 10:55:24,026 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-04 10:55:24,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,026 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-04 10:55:24,026 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-04 10:55:24,026 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-04 10:55:24,026 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-04 10:55:24,026 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-04 10:55:24,026 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:24,026 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-04 10:55:24,026 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-04 10:55:24,027 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-04 10:55:24,027 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-04 10:55:24,027 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-04 10:55:24,027 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:24,027 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-04 10:55:24,027 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-04 10:55:24,027 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-04 10:55:24,027 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-04 10:55:24,027 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-04 10:55:24,027 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:24,027 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-04 10:55:24,027 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-04 10:55:24,027 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:24,027 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:24,027 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:24,027 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. started... 2023-05-04 10:55:24,027 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bb802aa4052fe7e449f1c59066b7ab3a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:24,028 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,028 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,029 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,029 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:24,029 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,029 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:24,037 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf1/e371f3610c3c41648a76e3308e794ed3 2023-05-04 10:55:24,051 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf2/42e95bc7b6e74c38af889768c4b6b214 2023-05-04 10:55:24,055 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf1/e371f3610c3c41648a76e3308e794ed3 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/e371f3610c3c41648a76e3308e794ed3 2023-05-04 10:55:24,058 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/e371f3610c3c41648a76e3308e794ed3, entries=10, sequenceid=45, filesize=5.0 K 2023-05-04 10:55:24,059 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf2/42e95bc7b6e74c38af889768c4b6b214 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/42e95bc7b6e74c38af889768c4b6b214 2023-05-04 10:55:24,063 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/42e95bc7b6e74c38af889768c4b6b214, entries=10, sequenceid=45, filesize=5.0 K 2023-05-04 10:55:24,064 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for bb802aa4052fe7e449f1c59066b7ab3a in 37ms, sequenceid=45, compaction requested=true 2023-05-04 10:55:24,064 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bb802aa4052fe7e449f1c59066b7ab3a: 2023-05-04 10:55:24,064 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:24,064 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:24,064 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:24,064 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,064 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-04 10:55:24,064 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-04 10:55:24,066 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,066 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:24,066 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,066 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:24,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:24,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:24,068 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-04 10:55:24,068 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@57cdc734[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:24,068 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-04 10:55:24,068 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,071 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,071 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,071 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,071 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:24,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,071 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,071 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-04 10:55:24,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,071 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-04 10:55:24,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,071 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:24,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,071 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-04 10:55:24,071 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-04 10:55:24,071 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-04 10:55:24,071 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-04 10:55:24,071 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:24,071 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:24,072 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-04 10:55:24,072 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:24,072 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,072 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-04 10:55:24,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:24,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:24,073 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-04 10:55:24,073 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:24,073 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-04 10:55:24,074 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-04 10:55:24,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:24,074 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:24,074 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-04 10:55:24,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:24,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:24,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:24,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:24,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33581,1683197625489': 2023-05-04 10:55:24,077 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-04 10:55:24,077 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-04 10:55:24,077 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:24,077 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:24,077 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-04 10:55:24,077 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:24,079 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,079 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,079 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:24,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:24,079 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,079 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:24,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,079 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:24,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:24,080 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:24,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:24,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:24,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,080 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:24,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:24,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:24,081 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:24,081 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:24,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:24,083 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,083 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,083 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,086 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,086 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:24,086 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,086 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:24,086 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:24,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:24,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:24,086 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:24,086 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-04 10:55:24,086 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:24,086 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:24,086 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:24,086 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:24,086 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:24,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:24,086 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:24,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:24,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:24,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,086 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:24,086 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,087 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:24,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:24,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:24,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,087 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,087 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,087 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,087 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,087 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,087 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,187 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:24,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:24,199 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:24,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:24,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-04 10:55:24,201 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-04 10:55:24,201 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:24,201 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-04 10:55:24,201 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-04 10:55:24,201 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,201 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,202 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:24,202 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:24,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:24,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:24,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:24,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:24,202 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,202 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:24,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:24,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:24,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,203 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,203 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:24,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,203 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,203 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,203 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:24,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,204 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-04 10:55:24,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:24,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,204 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-04 10:55:24,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:24,204 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-04 10:55:24,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,204 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-04 10:55:24,204 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-04 10:55:24,204 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-04 10:55:24,204 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-04 10:55:24,204 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-04 10:55:24,204 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-04 10:55:24,204 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:24,204 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:24,204 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-04 10:55:24,204 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-04 10:55:24,204 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-04 10:55:24,204 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-04 10:55:24,204 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-04 10:55:24,204 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:24,204 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:24,204 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:24,204 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-04 10:55:24,205 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. started... 2023-05-04 10:55:24,205 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-04 10:55:24,205 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:24,205 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-04 10:55:24,205 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-04 10:55:24,205 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bb802aa4052fe7e449f1c59066b7ab3a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:24,205 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-04 10:55:24,206 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,207 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,207 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:24,207 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,207 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,207 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:24,214 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf1/f930bba2907a4f73a8dbead5885ae8ae 2023-05-04 10:55:24,225 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf2/8625a396e8374b85b74c8437daf17cbf 2023-05-04 10:55:24,229 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf1/f930bba2907a4f73a8dbead5885ae8ae as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/f930bba2907a4f73a8dbead5885ae8ae 2023-05-04 10:55:24,232 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/f930bba2907a4f73a8dbead5885ae8ae, entries=10, sequenceid=58, filesize=5.0 K 2023-05-04 10:55:24,233 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf2/8625a396e8374b85b74c8437daf17cbf as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/8625a396e8374b85b74c8437daf17cbf 2023-05-04 10:55:24,236 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/8625a396e8374b85b74c8437daf17cbf, entries=10, sequenceid=58, filesize=5.0 K 2023-05-04 10:55:24,237 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for bb802aa4052fe7e449f1c59066b7ab3a in 32ms, sequenceid=58, compaction requested=true 2023-05-04 10:55:24,237 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bb802aa4052fe7e449f1c59066b7ab3a: 2023-05-04 10:55:24,237 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:24,237 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:24,237 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:24,237 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,237 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-04 10:55:24,237 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-04 10:55:24,240 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,240 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:24,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:24,240 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,240 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:24,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:24,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:24,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:24,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:24,242 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-04 10:55:24,242 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ea4e362[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:24,242 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-04 10:55:24,242 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,243 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,243 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,243 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,243 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-04 10:55:24,243 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,243 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:24,243 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-04 10:55:24,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,243 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-04 10:55:24,244 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-04 10:55:24,244 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-04 10:55:24,243 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:24,244 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:24,244 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-04 10:55:24,244 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:24,248 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-04 10:55:24,248 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:24,248 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-04 10:55:24,248 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:24,248 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-04 10:55:24,248 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-04 10:55:24,257 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-04 10:55:24,257 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:24,257 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,257 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-04 10:55:24,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:24,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:24,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:24,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:24,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:24,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:24,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:24,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33581,1683197625489': 2023-05-04 10:55:24,261 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-04 10:55:24,261 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-04 10:55:24,261 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:24,261 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:24,261 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-04 10:55:24,261 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:24,262 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,262 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,262 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,262 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:24,262 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:24,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:24,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,262 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:24,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,263 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,263 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:24,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:24,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:24,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:24,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,263 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:24,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:24,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,264 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:24,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:24,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,265 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:24,266 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:24,266 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,268 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:24,268 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:24,268 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:24,269 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-04 10:55:24,269 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:24,268 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:24,268 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:24,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:24,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:24,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:24,268 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:24,269 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:24,268 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,269 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:24,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:24,269 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:24,269 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:24,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:24,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:24,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:24,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,269 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,270 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,270 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,270 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,270 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:24,270 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,270 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,270 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,270 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,369 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:24,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:24,382 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:24,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:24,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-04 10:55:24,383 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-04 10:55:24,383 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:24,383 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-04 10:55:24,383 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-04 10:55:24,384 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,384 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,386 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:24,386 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:24,386 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:24,386 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:24,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:24,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:24,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:24,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:24,386 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,386 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:24,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:24,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,387 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,387 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,387 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:24,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:24,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,387 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-04 10:55:24,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:24,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,387 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-04 10:55:24,387 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-04 10:55:24,387 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-04 10:55:24,387 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-04 10:55:24,388 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-04 10:55:24,388 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-04 10:55:24,387 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-04 10:55:24,388 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:24,388 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-04 10:55:24,388 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-04 10:55:24,388 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-04 10:55:24,388 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:24,388 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-04 10:55:24,388 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-04 10:55:24,388 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-04 10:55:24,388 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:24,388 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-04 10:55:24,388 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-04 10:55:24,388 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-04 10:55:24,388 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-04 10:55:24,388 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-04 10:55:24,388 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:24,388 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:24,388 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:24,388 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. started... 2023-05-04 10:55:24,388 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bb802aa4052fe7e449f1c59066b7ab3a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:24,389 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,390 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,390 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,390 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,390 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:24,390 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:24,397 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf1/4b53c691f32543299b09428b4a503ff6 2023-05-04 10:55:24,408 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf2/a3e671966b44448b8e8ed3ca6e8f9a32 2023-05-04 10:55:24,413 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf1/4b53c691f32543299b09428b4a503ff6 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/4b53c691f32543299b09428b4a503ff6 2023-05-04 10:55:24,416 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/4b53c691f32543299b09428b4a503ff6, entries=10, sequenceid=71, filesize=5.0 K 2023-05-04 10:55:24,417 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf2/a3e671966b44448b8e8ed3ca6e8f9a32 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/a3e671966b44448b8e8ed3ca6e8f9a32 2023-05-04 10:55:24,421 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/a3e671966b44448b8e8ed3ca6e8f9a32, entries=10, sequenceid=71, filesize=5.0 K 2023-05-04 10:55:24,421 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for bb802aa4052fe7e449f1c59066b7ab3a in 33ms, sequenceid=71, compaction requested=true 2023-05-04 10:55:24,421 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bb802aa4052fe7e449f1c59066b7ab3a: 2023-05-04 10:55:24,422 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:24,422 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:24,422 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:24,422 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,422 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-04 10:55:24,422 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-04 10:55:24,423 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,423 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:24,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:24,423 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:24,423 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:24,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:24,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:24,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:24,425 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-04 10:55:24,425 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b232db2[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:24,425 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-04 10:55:24,425 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,426 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,426 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,426 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,426 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:24,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,426 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,426 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-04 10:55:24,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,427 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-04 10:55:24,427 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-04 10:55:24,427 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-04 10:55:24,427 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-04 10:55:24,427 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:24,427 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:24,427 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-04 10:55:24,427 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:24,429 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,429 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-04 10:55:24,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:24,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:24,429 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:24,429 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-04 10:55:24,430 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:24,429 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-04 10:55:24,430 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-04 10:55:24,430 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-04 10:55:24,430 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:24,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:24,430 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-04 10:55:24,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:24,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:24,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:24,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:24,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33581,1683197625489': 2023-05-04 10:55:24,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-04 10:55:24,433 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-04 10:55:24,433 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:24,433 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:24,433 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-04 10:55:24,433 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:24,434 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,434 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,434 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:24,434 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:24,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,434 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:24,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:24,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,434 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,434 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:24,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:24,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:24,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:24,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,435 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:24,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:24,435 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:24,435 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:24,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:24,437 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-04 10:55:24,437 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,438 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,440 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,440 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,440 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:24,440 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:24,440 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:24,440 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:24,440 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,440 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:24,440 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:24,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:24,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:24,440 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:24,440 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-04 10:55:24,440 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-04 10:55:24,440 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:24,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:24,440 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:24,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:24,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:24,440 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:24,441 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,441 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:24,441 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,441 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:24,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:24,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,441 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:24,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:24,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,441 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-04 10:55:24,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:24,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:24,441 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-04 10:55:24,541 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:24,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:24,549 INFO [Listener at localhost/35483] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,551 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33581] regionserver.StoreUtils(129): cannot split hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/cd571975264247f2b6e28230994f891b because midkey is the same as first or last row 2023-05-04 10:55:24,551 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33581] regionserver.StoreUtils(129): cannot split hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/ac94f4093c1b4ee3bed720a959838d80 because midkey is the same as first or last row 2023-05-04 10:55:24,551 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33581] regionserver.HRegion(2745): Flushing bb802aa4052fe7e449f1c59066b7ab3a 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-04 10:55:24,559 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33581] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf1/c096b9557028480eb7923bef203758da 2023-05-04 10:55:24,570 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33581] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf2/07bbd060250840d38286b3f47f4d50f4 2023-05-04 10:55:24,574 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33581] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf1/c096b9557028480eb7923bef203758da as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/c096b9557028480eb7923bef203758da 2023-05-04 10:55:24,577 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33581] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/c096b9557028480eb7923bef203758da, entries=8, sequenceid=82, filesize=5.0 K 2023-05-04 10:55:24,578 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33581] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/.tmp/cf2/07bbd060250840d38286b3f47f4d50f4 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/07bbd060250840d38286b3f47f4d50f4 2023-05-04 10:55:24,582 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33581] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/07bbd060250840d38286b3f47f4d50f4, entries=8, sequenceid=82, filesize=5.0 K 2023-05-04 10:55:24,582 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33581] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for bb802aa4052fe7e449f1c59066b7ab3a in 31ms, sequenceid=82, compaction requested=true 2023-05-04 10:55:24,583 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33581] regionserver.HRegion(2446): Flush status journal for bb802aa4052fe7e449f1c59066b7ab3a: 2023-05-04 10:55:24,583 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33581] regionserver.StoreUtils(129): cannot split hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/cd571975264247f2b6e28230994f891b because midkey is the same as first or last row 2023-05-04 10:55:24,583 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33581] regionserver.StoreUtils(129): cannot split hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/ac94f4093c1b4ee3bed720a959838d80 because midkey is the same as first or last row 2023-05-04 10:55:24,584 DEBUG [Listener at localhost/35483] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,584 DEBUG [Listener at localhost/35483] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-05-04 10:55:24,584 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:55:24,585 INFO [Listener at localhost/35483] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-05-04 10:55:24,586 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-05-04 10:55:24,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:24,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-05-04 10:55:24,589 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197724589"}]},"ts":"1683197724589"} 2023-05-04 10:55:24,590 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-05-04 10:55:24,592 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-05-04 10:55:24,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=bb802aa4052fe7e449f1c59066b7ab3a, UNASSIGN}] 2023-05-04 10:55:24,593 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=bb802aa4052fe7e449f1c59066b7ab3a, UNASSIGN 2023-05-04 10:55:24,594 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=bb802aa4052fe7e449f1c59066b7ab3a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:24,594 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1683197724594"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197724594"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197724594"}]},"ts":"1683197724594"} 2023-05-04 10:55:24,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure bb802aa4052fe7e449f1c59066b7ab3a, server=jenkins-hbase4.apache.org,33581,1683197625489}] 2023-05-04 10:55:24,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-05-04 10:55:24,747 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close bb802aa4052fe7e449f1c59066b7ab3a 2023-05-04 10:55:24,747 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bb802aa4052fe7e449f1c59066b7ab3a, disabling compactions & flushes 2023-05-04 10:55:24,747 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:24,747 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:24,747 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. after waiting 0 ms 2023-05-04 10:55:24,747 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:24,751 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-04 10:55:24,751 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:55:24,752 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a. 2023-05-04 10:55:24,752 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bb802aa4052fe7e449f1c59066b7ab3a: 2023-05-04 10:55:24,753 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed bb802aa4052fe7e449f1c59066b7ab3a 2023-05-04 10:55:24,753 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=bb802aa4052fe7e449f1c59066b7ab3a, regionState=CLOSED 2023-05-04 10:55:24,754 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1683197724753"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197724753"}]},"ts":"1683197724753"} 2023-05-04 10:55:24,756 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-05-04 10:55:24,756 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure bb802aa4052fe7e449f1c59066b7ab3a, server=jenkins-hbase4.apache.org,33581,1683197625489 in 159 msec 2023-05-04 10:55:24,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-05-04 10:55:24,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=bb802aa4052fe7e449f1c59066b7ab3a, UNASSIGN in 163 msec 2023-05-04 10:55:24,758 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197724758"}]},"ts":"1683197724758"} 2023-05-04 10:55:24,759 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-05-04 10:55:24,760 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-05-04 10:55:24,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 175 msec 2023-05-04 10:55:24,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-05-04 10:55:24,891 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-05-04 10:55:24,892 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-05-04 10:55:24,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:24,894 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:24,895 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:24,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-05-04 10:55:24,898 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a 2023-05-04 10:55:24,900 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/recovered.edits] 2023-05-04 10:55:24,903 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/4b53c691f32543299b09428b4a503ff6 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/4b53c691f32543299b09428b4a503ff6 2023-05-04 10:55:24,905 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/bd80b0760ee94762a162655715963a02 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/bd80b0760ee94762a162655715963a02 2023-05-04 10:55:24,906 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/c096b9557028480eb7923bef203758da to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/c096b9557028480eb7923bef203758da 2023-05-04 10:55:24,907 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/cd571975264247f2b6e28230994f891b to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/cd571975264247f2b6e28230994f891b 2023-05-04 10:55:24,908 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/d9f843e079fc41b8b97b52dbd3458512 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/d9f843e079fc41b8b97b52dbd3458512 2023-05-04 10:55:24,909 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/e371f3610c3c41648a76e3308e794ed3 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/e371f3610c3c41648a76e3308e794ed3 2023-05-04 10:55:24,910 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/f930bba2907a4f73a8dbead5885ae8ae to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf1/f930bba2907a4f73a8dbead5885ae8ae 2023-05-04 10:55:24,912 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/07bbd060250840d38286b3f47f4d50f4 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/07bbd060250840d38286b3f47f4d50f4 2023-05-04 10:55:24,913 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/2bd92ed91bad4ef1a754484d24521173 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/2bd92ed91bad4ef1a754484d24521173 2023-05-04 10:55:24,914 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/42e95bc7b6e74c38af889768c4b6b214 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/42e95bc7b6e74c38af889768c4b6b214 2023-05-04 10:55:24,915 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/8625a396e8374b85b74c8437daf17cbf to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/8625a396e8374b85b74c8437daf17cbf 2023-05-04 10:55:24,916 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/a3e671966b44448b8e8ed3ca6e8f9a32 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/a3e671966b44448b8e8ed3ca6e8f9a32 2023-05-04 10:55:24,917 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/a8b9b0641a7e445daf5fb493a236b60e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/a8b9b0641a7e445daf5fb493a236b60e 2023-05-04 10:55:24,918 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/ac94f4093c1b4ee3bed720a959838d80 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/cf2/ac94f4093c1b4ee3bed720a959838d80 2023-05-04 10:55:24,920 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/recovered.edits/85.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a/recovered.edits/85.seqid 2023-05-04 10:55:24,921 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testInvalidSplitKey/bb802aa4052fe7e449f1c59066b7ab3a 2023-05-04 10:55:24,921 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-05-04 10:55:24,923 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:24,924 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-05-04 10:55:24,928 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-05-04 10:55:24,929 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:24,929 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-05-04 10:55:24,929 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197724929"}]},"ts":"9223372036854775807"} 2023-05-04 10:55:24,930 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-05-04 10:55:24,930 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => bb802aa4052fe7e449f1c59066b7ab3a, NAME => 'testInvalidSplitKey,,1683197722928.bb802aa4052fe7e449f1c59066b7ab3a.', STARTKEY => '', ENDKEY => ''}] 2023-05-04 10:55:24,930 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-05-04 10:55:24,930 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683197724930"}]},"ts":"9223372036854775807"} 2023-05-04 10:55:24,931 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-05-04 10:55:24,933 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:24,934 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 41 msec 2023-05-04 10:55:24,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-05-04 10:55:24,997 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-05-04 10:55:25,010 INFO [Listener at localhost/35483] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=369 (was 348) - Thread LEAK? -, OpenFileDescriptor=742 (was 655) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=181 (was 189), ProcessCount=170 (was 170), AvailableMemoryMB=6328 (was 6380) 2023-05-04 10:55:25,023 INFO [Listener at localhost/35483] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=369, OpenFileDescriptor=742, MaxFileDescriptor=60000, SystemLoadAverage=181, ProcessCount=170, AvailableMemoryMB=6327 2023-05-04 10:55:25,023 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:55:25,025 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-04 10:55:25,025 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-04 10:55:25,027 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:25,028 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-05-04 10:55:25,029 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-04 10:55:25,030 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:25,031 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53 empty. 2023-05-04 10:55:25,031 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:25,031 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-05-04 10:55:25,040 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-05-04 10:55:25,041 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => ae8b725404c1d45e9f47aaaa71304e53, NAME => 'testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp 2023-05-04 10:55:25,049 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:55:25,049 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing ae8b725404c1d45e9f47aaaa71304e53, disabling compactions & flushes 2023-05-04 10:55:25,049 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:25,049 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:25,049 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. after waiting 0 ms 2023-05-04 10:55:25,049 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:25,049 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:25,049 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for ae8b725404c1d45e9f47aaaa71304e53: 2023-05-04 10:55:25,051 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-05-04 10:55:25,052 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197725052"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197725052"}]},"ts":"1683197725052"} 2023-05-04 10:55:25,053 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-04 10:55:25,054 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-04 10:55:25,054 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197725054"}]},"ts":"1683197725054"} 2023-05-04 10:55:25,055 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-05-04 10:55:25,058 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-04 10:55:25,059 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-04 10:55:25,059 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-04 10:55:25,059 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-04 10:55:25,059 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-04 10:55:25,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, ASSIGN}] 2023-05-04 10:55:25,060 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, ASSIGN 2023-05-04 10:55:25,061 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37343,1683197625438; forceNewPlan=false, retain=false 2023-05-04 10:55:25,089 ERROR [BP-2139253951-172.31.14.131-1683197620399 heartbeating to localhost/127.0.0.1:44245] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2139253951-172.31.14.131-1683197620399 (Datanode Uuid 9432e34f-f582-4227-a602-1b8f52f4fa10) service to localhost/127.0.0.1:44245 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-05-04 10:55:25,211 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-04 10:55:25,212 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=ae8b725404c1d45e9f47aaaa71304e53, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,213 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197725212"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197725212"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197725212"}]},"ts":"1683197725212"} 2023-05-04 10:55:25,214 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438}] 2023-05-04 10:55:25,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:25,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ae8b725404c1d45e9f47aaaa71304e53, NAME => 'testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53.', STARTKEY => '', ENDKEY => ''} 2023-05-04 10:55:25,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:55:25,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:25,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:55:25,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:25,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:25,372 INFO [StoreOpener-ae8b725404c1d45e9f47aaaa71304e53-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:25,373 DEBUG [StoreOpener-ae8b725404c1d45e9f47aaaa71304e53-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1 2023-05-04 10:55:25,373 DEBUG [StoreOpener-ae8b725404c1d45e9f47aaaa71304e53-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1 2023-05-04 10:55:25,374 INFO [StoreOpener-ae8b725404c1d45e9f47aaaa71304e53-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ae8b725404c1d45e9f47aaaa71304e53 columnFamilyName cf1 2023-05-04 10:55:25,374 INFO [StoreOpener-ae8b725404c1d45e9f47aaaa71304e53-1] regionserver.HStore(310): Store=ae8b725404c1d45e9f47aaaa71304e53/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:25,374 INFO [StoreOpener-ae8b725404c1d45e9f47aaaa71304e53-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:25,375 DEBUG [StoreOpener-ae8b725404c1d45e9f47aaaa71304e53-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2 2023-05-04 10:55:25,375 DEBUG [StoreOpener-ae8b725404c1d45e9f47aaaa71304e53-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2 2023-05-04 10:55:25,375 INFO [StoreOpener-ae8b725404c1d45e9f47aaaa71304e53-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ae8b725404c1d45e9f47aaaa71304e53 columnFamilyName cf2 2023-05-04 10:55:25,376 INFO [StoreOpener-ae8b725404c1d45e9f47aaaa71304e53-1] regionserver.HStore(310): Store=ae8b725404c1d45e9f47aaaa71304e53/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:25,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:25,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:25,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:55:25,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:25,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-04 10:55:25,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ae8b725404c1d45e9f47aaaa71304e53; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69059662, jitterRate=0.02906915545463562}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:55:25,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ae8b725404c1d45e9f47aaaa71304e53: 2023-05-04 10:55:25,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53., pid=111, masterSystemTime=1683197725366 2023-05-04 10:55:25,384 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:25,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:25,384 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=ae8b725404c1d45e9f47aaaa71304e53, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,385 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197725384"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197725384"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197725384"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197725384"}]},"ts":"1683197725384"} 2023-05-04 10:55:25,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-05-04 10:55:25,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438 in 172 msec 2023-05-04 10:55:25,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-05-04 10:55:25,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, ASSIGN in 328 msec 2023-05-04 10:55:25,389 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-04 10:55:25,389 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197725389"}]},"ts":"1683197725389"} 2023-05-04 10:55:25,390 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-05-04 10:55:25,392 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-05-04 10:55:25,393 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 366 msec 2023-05-04 10:55:25,533 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:25,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-04 10:55:25,535 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-04 10:55:25,535 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:25,535 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-04 10:55:25,535 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-04 10:55:25,536 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,536 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,537 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:25,537 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:25,537 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:25,537 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:25,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:25,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:25,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:25,537 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,537 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:25,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:25,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:25,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,538 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,538 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,538 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:25,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:25,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:25,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,538 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-04 10:55:25,538 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-04 10:55:25,538 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-04 10:55:25,538 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-04 10:55:25,539 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-04 10:55:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-04 10:55:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-04 10:55:25,539 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-04 10:55:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-04 10:55:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:55:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:55:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:55:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-04 10:55:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-04 10:55:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:25,539 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-04 10:55:25,539 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. started... 2023-05-04 10:55:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-04 10:55:25,542 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ae8b725404c1d45e9f47aaaa71304e53 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-04 10:55:25,544 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,544 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,544 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,544 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:25,544 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,544 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:25,554 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf1/8b785c69c5ce402ea7e230aa15f2dd2b 2023-05-04 10:55:25,570 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf2/659180a757a544d5858aec0baae07501 2023-05-04 10:55:25,574 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf1/8b785c69c5ce402ea7e230aa15f2dd2b as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/8b785c69c5ce402ea7e230aa15f2dd2b 2023-05-04 10:55:25,577 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/8b785c69c5ce402ea7e230aa15f2dd2b, entries=2, sequenceid=6, filesize=4.8 K 2023-05-04 10:55:25,578 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf2/659180a757a544d5858aec0baae07501 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/659180a757a544d5858aec0baae07501 2023-05-04 10:55:25,581 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/659180a757a544d5858aec0baae07501, entries=2, sequenceid=6, filesize=4.8 K 2023-05-04 10:55:25,582 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ae8b725404c1d45e9f47aaaa71304e53 in 41ms, sequenceid=6, compaction requested=false 2023-05-04 10:55:25,582 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-05-04 10:55:25,582 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ae8b725404c1d45e9f47aaaa71304e53: 2023-05-04 10:55:25,583 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:25,583 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:25,583 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:25,583 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,583 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-04 10:55:25,583 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-04 10:55:25,584 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,584 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:25,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:25,584 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,584 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:25,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:25,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:25,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:25,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:25,586 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-04 10:55:25,586 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5518e667[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:25,586 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-04 10:55:25,586 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,587 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,587 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,587 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,587 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:55:25,588 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,588 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:25,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,588 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-04 10:55:25,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,588 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:25,588 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:55:25,588 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-04 10:55:25,588 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:25,588 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:55:25,588 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-04 10:55:25,588 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:25,589 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-04 10:55:25,589 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:25,589 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-04 10:55:25,591 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-04 10:55:25,591 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-04 10:55:25,591 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:25,591 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:25,591 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-04 10:55:25,591 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,591 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-04 10:55:25,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:25,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:25,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:25,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:25,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:25,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:25,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:25,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,37343,1683197625438': 2023-05-04 10:55:25,594 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-04 10:55:25,594 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-04 10:55:25,594 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:25,594 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:25,594 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-04 10:55:25,594 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:25,595 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,595 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:25,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:25,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,595 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:25,595 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,595 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,596 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:25,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,596 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:25,596 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:25,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:25,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:25,597 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:25,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:25,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:25,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,598 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:25,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:25,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:25,599 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,600 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,600 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,602 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,602 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,602 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:25,602 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,602 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:25,602 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:25,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:25,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:25,602 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:25,602 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,602 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:25,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:25,602 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:25,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:25,602 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,602 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-04 10:55:25,602 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,603 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:25,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:25,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:25,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:25,602 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:25,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:25,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:25,603 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:25,603 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,603 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:25,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:25,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:25,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:25,603 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,603 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,603 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,603 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,703 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:25,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:25,715 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:25,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:25,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-04 10:55:25,717 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-04 10:55:25,717 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:25,717 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-04 10:55:25,717 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-04 10:55:25,717 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,717 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,718 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:25,718 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:25,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:25,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:25,718 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,718 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:25,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:25,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:25,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:25,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:25,719 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,719 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:25,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,719 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,719 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,719 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:25,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,720 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-04 10:55:25,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:25,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:25,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,720 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-04 10:55:25,720 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-04 10:55:25,720 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-04 10:55:25,720 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-04 10:55:25,720 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-04 10:55:25,720 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:25,720 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:55:25,720 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-04 10:55:25,720 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-04 10:55:25,720 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-04 10:55:25,720 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-04 10:55:25,721 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-04 10:55:25,721 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-04 10:55:25,721 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:25,721 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-04 10:55:25,721 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:55:25,721 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-04 10:55:25,721 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:25,721 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:55:25,721 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-04 10:55:25,721 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-04 10:55:25,721 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:25,721 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:25,721 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:25,721 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. started... 2023-05-04 10:55:25,721 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ae8b725404c1d45e9f47aaaa71304e53 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:25,723 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,724 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,724 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,724 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:25,724 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,724 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:25,731 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf1/485bec54ae1147ff854248d9b1f4ce44 2023-05-04 10:55:25,741 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf2/ec68aac4dd8b426f9f4503314c0964be 2023-05-04 10:55:25,745 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf1/485bec54ae1147ff854248d9b1f4ce44 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/485bec54ae1147ff854248d9b1f4ce44 2023-05-04 10:55:25,749 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/485bec54ae1147ff854248d9b1f4ce44, entries=10, sequenceid=19, filesize=5.0 K 2023-05-04 10:55:25,750 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf2/ec68aac4dd8b426f9f4503314c0964be as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/ec68aac4dd8b426f9f4503314c0964be 2023-05-04 10:55:25,754 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/ec68aac4dd8b426f9f4503314c0964be, entries=10, sequenceid=19, filesize=5.0 K 2023-05-04 10:55:25,754 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ae8b725404c1d45e9f47aaaa71304e53 in 33ms, sequenceid=19, compaction requested=false 2023-05-04 10:55:25,754 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ae8b725404c1d45e9f47aaaa71304e53: 2023-05-04 10:55:25,755 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:25,755 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:25,755 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:25,755 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,755 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-04 10:55:25,755 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-04 10:55:25,759 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,759 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:25,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:25,760 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,760 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:25,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:25,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:25,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:25,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:25,761 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-04 10:55:25,762 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c557f97[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:25,762 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-04 10:55:25,762 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,764 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,764 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,764 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,764 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:55:25,764 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-04 10:55:25,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,764 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,764 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:55:25,764 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-04 10:55:25,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,765 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:25,765 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:55:25,765 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-04 10:55:25,765 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:25,764 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:25,764 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-04 10:55:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-04 10:55:25,766 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:25,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:25,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:25,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-04 10:55:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-04 10:55:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-04 10:55:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-04 10:55:25,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:25,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:25,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:25,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:25,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:25,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,37343,1683197625438': 2023-05-04 10:55:25,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-04 10:55:25,769 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-04 10:55:25,769 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:25,770 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:25,770 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-04 10:55:25,770 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:25,771 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,771 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,771 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:25,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:25,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,771 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:25,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,771 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:25,771 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,771 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:25,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:25,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:25,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:25,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:25,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:25,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,771 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:25,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:25,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:25,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:25,772 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:25,773 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:25,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:25,774 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,774 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,774 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,777 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,777 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:25,777 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:25,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:25,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:25,777 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:25,777 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:25,777 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,777 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:25,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:25,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:25,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:25,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:25,777 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:25,777 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:25,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:25,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:25,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:25,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:25,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:25,778 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,778 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,778 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,778 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:25,778 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-04 10:55:25,778 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:25,778 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,778 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,778 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,778 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,778 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,778 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:25,778 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:25,878 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:25,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:25,891 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:25,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:25,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-04 10:55:25,892 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-04 10:55:25,893 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:25,893 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-04 10:55:25,893 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-04 10:55:25,893 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,893 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,895 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:25,895 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:25,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:25,895 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:25,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:25,895 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:25,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:25,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:25,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:25,895 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,895 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:25,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,896 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,896 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:25,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:25,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,896 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-04 10:55:25,896 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-04 10:55:25,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,896 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-04 10:55:25,897 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-04 10:55:25,897 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-04 10:55:25,897 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-04 10:55:25,897 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:25,897 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,897 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-04 10:55:25,897 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:55:25,897 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-04 10:55:25,897 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:25,897 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:55:25,897 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-04 10:55:25,897 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-04 10:55:25,897 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:25,897 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:25,897 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:25,897 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. started... 2023-05-04 10:55:25,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:25,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,897 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ae8b725404c1d45e9f47aaaa71304e53 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:25,897 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-04 10:55:25,898 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-04 10:55:25,898 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-04 10:55:25,898 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-04 10:55:25,898 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:25,898 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:55:25,898 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-04 10:55:25,898 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-04 10:55:25,898 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,899 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,899 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,899 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:25,899 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,900 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:25,910 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf1/494a5629975e4feb90318ed90245269b 2023-05-04 10:55:25,922 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf2/1f022c1ccf24485e99e47cc107060387 2023-05-04 10:55:25,926 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf1/494a5629975e4feb90318ed90245269b as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/494a5629975e4feb90318ed90245269b 2023-05-04 10:55:25,930 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/494a5629975e4feb90318ed90245269b, entries=10, sequenceid=32, filesize=5.0 K 2023-05-04 10:55:25,931 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf2/1f022c1ccf24485e99e47cc107060387 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/1f022c1ccf24485e99e47cc107060387 2023-05-04 10:55:25,934 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/1f022c1ccf24485e99e47cc107060387, entries=10, sequenceid=32, filesize=5.0 K 2023-05-04 10:55:25,935 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ae8b725404c1d45e9f47aaaa71304e53 in 38ms, sequenceid=32, compaction requested=true 2023-05-04 10:55:25,935 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ae8b725404c1d45e9f47aaaa71304e53: 2023-05-04 10:55:25,935 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:25,935 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:25,935 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:25,936 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,936 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-04 10:55:25,936 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-04 10:55:25,937 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,937 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:25,937 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,937 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:25,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:25,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:25,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:25,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:25,939 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-04 10:55:25,939 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5832e8c[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:25,939 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-04 10:55:25,940 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,942 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,942 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,942 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,942 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:55:25,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,942 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,942 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:55:25,942 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-04 10:55:25,942 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-04 10:55:25,942 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:25,942 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:25,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,942 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:25,942 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:55:25,942 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-04 10:55:25,942 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:25,944 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,944 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-04 10:55:25,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:25,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:25,944 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-04 10:55:25,944 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:25,944 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:25,944 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-04 10:55:25,944 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-04 10:55:25,944 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-04 10:55:25,944 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:25,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:25,944 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-04 10:55:25,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:25,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:25,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:25,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:25,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,37343,1683197625438': 2023-05-04 10:55:25,947 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-04 10:55:25,947 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-04 10:55:25,947 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:25,947 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:25,947 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-04 10:55:25,947 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:25,949 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,949 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,949 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:25,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:25,949 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,949 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,949 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:25,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,949 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:25,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:25,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:25,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:25,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:25,950 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:25,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:25,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:25,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:25,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,950 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:25,951 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:25,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:25,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:25,952 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:25,953 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,953 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,962 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,962 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:25,962 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:25,962 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:25,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:25,963 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:25,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:25,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:25,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:25,963 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:25,963 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:25,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:25,963 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:25,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:25,963 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,963 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:25,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:25,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:25,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:25,963 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-04 10:55:25,963 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:25,963 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:25,963 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:25,964 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:25,964 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,964 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:25,964 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:25,964 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,964 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:25,964 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:25,964 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:26,064 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:26,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:26,077 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:26,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:26,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-04 10:55:26,078 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-04 10:55:26,078 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:26,078 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-04 10:55:26,078 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-04 10:55:26,079 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,079 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,080 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,080 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:26,080 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:26,080 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:26,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:26,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:26,080 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,080 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:26,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:26,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:26,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:26,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:26,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:26,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:26,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:26,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,082 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-04 10:55:26,082 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-04 10:55:26,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-04 10:55:26,082 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-04 10:55:26,082 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-04 10:55:26,082 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-04 10:55:26,082 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:26,082 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-04 10:55:26,082 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:55:26,082 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-04 10:55:26,082 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-04 10:55:26,082 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-04 10:55:26,082 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-04 10:55:26,082 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:26,082 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:55:26,082 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-04 10:55:26,082 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-04 10:55:26,083 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,084 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-04 10:55:26,084 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-04 10:55:26,084 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,084 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,084 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:26,084 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-04 10:55:26,084 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:26,084 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:55:26,084 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,084 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:26,084 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:26,084 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:26,085 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:26,085 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. started... 2023-05-04 10:55:26,085 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ae8b725404c1d45e9f47aaaa71304e53 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:26,098 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf1/17b163b7ed58427cb3d0eff92a1687e6 2023-05-04 10:55:26,109 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf2/724e838af6de4b788c2d4e879968d8df 2023-05-04 10:55:26,113 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf1/17b163b7ed58427cb3d0eff92a1687e6 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/17b163b7ed58427cb3d0eff92a1687e6 2023-05-04 10:55:26,116 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/17b163b7ed58427cb3d0eff92a1687e6, entries=10, sequenceid=45, filesize=5.0 K 2023-05-04 10:55:26,117 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf2/724e838af6de4b788c2d4e879968d8df as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/724e838af6de4b788c2d4e879968d8df 2023-05-04 10:55:26,121 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/724e838af6de4b788c2d4e879968d8df, entries=10, sequenceid=45, filesize=5.0 K 2023-05-04 10:55:26,122 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ae8b725404c1d45e9f47aaaa71304e53 in 37ms, sequenceid=45, compaction requested=true 2023-05-04 10:55:26,122 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ae8b725404c1d45e9f47aaaa71304e53: 2023-05-04 10:55:26,122 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:26,122 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:26,122 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:26,122 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,123 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-04 10:55:26,123 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-04 10:55:26,125 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,125 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:26,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:26,125 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,125 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:26,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:26,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:26,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:26,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:26,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-04 10:55:26,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17dfcd33[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:26,127 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-04 10:55:26,127 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,128 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,128 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,128 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,129 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,129 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:26,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,129 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:55:26,129 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:55:26,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,129 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-04 10:55:26,129 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:55:26,129 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-04 10:55:26,129 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:26,129 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-04 10:55:26,129 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:26,129 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:26,130 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-04 10:55:26,130 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:26,130 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-04 10:55:26,131 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-04 10:55:26,131 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-04 10:55:26,131 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:26,131 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,131 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-04 10:55:26,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:26,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:26,131 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:26,131 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-04 10:55:26,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:26,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:26,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:26,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:26,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:26,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,37343,1683197625438': 2023-05-04 10:55:26,134 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-04 10:55:26,134 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-04 10:55:26,134 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:26,134 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:26,134 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-04 10:55:26,134 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:26,136 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,136 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:26,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:26,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,136 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,136 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:26,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,136 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:26,137 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,136 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:26,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,137 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:26,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:26,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:26,137 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:26,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,137 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:26,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:26,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:26,139 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,139 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,140 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,142 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,142 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:26,142 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:26,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:26,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:26,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:26,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:26,142 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,142 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-04 10:55:26,142 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,142 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:26,142 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:26,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:26,142 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:26,142 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,142 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,142 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:26,143 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,142 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:26,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:26,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:26,142 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:26,143 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:26,143 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:26,143 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,143 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:26,143 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:26,143 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,143 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:26,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:26,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:26,243 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:26,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:26,255 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:26,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:26,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-04 10:55:26,257 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-04 10:55:26,257 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:26,257 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-04 10:55:26,257 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-04 10:55:26,258 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,258 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,259 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:26,259 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:26,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:26,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:26,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:26,259 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,259 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:26,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:26,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:26,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:26,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,259 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,259 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:26,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,259 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:26,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:26,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,260 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-04 10:55:26,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:26,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,260 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-04 10:55:26,260 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-04 10:55:26,260 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-04 10:55:26,260 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-04 10:55:26,260 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-04 10:55:26,260 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-04 10:55:26,260 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:26,261 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-04 10:55:26,261 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:55:26,261 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-04 10:55:26,261 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-04 10:55:26,261 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-04 10:55:26,261 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:26,261 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-04 10:55:26,261 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-04 10:55:26,261 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-04 10:55:26,261 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:26,261 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:55:26,261 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:55:26,261 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-04 10:55:26,261 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-04 10:55:26,261 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:26,261 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:26,261 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:26,261 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. started... 2023-05-04 10:55:26,261 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ae8b725404c1d45e9f47aaaa71304e53 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:26,262 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,264 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,264 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,264 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:26,264 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,264 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:26,270 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf1/2ed8f38d4ec34201af96d9bd26a06832 2023-05-04 10:55:26,281 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf2/95c03e8f9e614998a9a57fa46315f3fe 2023-05-04 10:55:26,284 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf1/2ed8f38d4ec34201af96d9bd26a06832 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/2ed8f38d4ec34201af96d9bd26a06832 2023-05-04 10:55:26,288 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/2ed8f38d4ec34201af96d9bd26a06832, entries=10, sequenceid=58, filesize=5.0 K 2023-05-04 10:55:26,288 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf2/95c03e8f9e614998a9a57fa46315f3fe as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/95c03e8f9e614998a9a57fa46315f3fe 2023-05-04 10:55:26,291 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/95c03e8f9e614998a9a57fa46315f3fe, entries=10, sequenceid=58, filesize=5.0 K 2023-05-04 10:55:26,292 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ae8b725404c1d45e9f47aaaa71304e53 in 31ms, sequenceid=58, compaction requested=true 2023-05-04 10:55:26,292 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ae8b725404c1d45e9f47aaaa71304e53: 2023-05-04 10:55:26,292 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:26,292 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:26,292 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:26,293 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,293 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-04 10:55:26,293 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-04 10:55:26,296 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,296 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:26,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:26,297 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,297 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:26,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:26,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:26,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:26,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:26,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-04 10:55:26,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5662a71[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:26,298 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-04 10:55:26,299 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,299 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,300 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,300 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:55:26,300 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-04 10:55:26,300 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,300 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:26,300 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,300 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:26,300 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:55:26,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,300 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-04 10:55:26,300 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:55:26,300 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-04 10:55:26,300 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:26,300 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:26,302 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-04 10:55:26,303 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:26,303 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-04 10:55:26,303 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-04 10:55:26,303 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:26,303 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-04 10:55:26,304 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-04 10:55:26,304 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:26,304 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,304 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-04 10:55:26,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:26,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:26,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:26,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:26,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:26,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:26,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:26,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,37343,1683197625438': 2023-05-04 10:55:26,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-04 10:55:26,307 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-04 10:55:26,307 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:26,307 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:26,307 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-04 10:55:26,307 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:26,309 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,309 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,309 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,309 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:26,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:26,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:26,309 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,309 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:26,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:26,309 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:26,309 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:26,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:26,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:26,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:26,310 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:26,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,310 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:26,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:26,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:26,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:26,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:26,311 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:26,312 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,312 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,315 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,315 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:26,315 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:26,315 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,315 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:26,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:26,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:26,315 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-04 10:55:26,315 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:26,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:26,315 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:26,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:26,316 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:26,315 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:26,315 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:26,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:26,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:26,315 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:26,315 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:26,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:26,316 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:26,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:26,316 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:26,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:26,316 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:26,317 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,317 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,317 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,317 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,317 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,317 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,417 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:26,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:26,429 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:26,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:26,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-04 10:55:26,431 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-04 10:55:26,431 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:26,431 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-04 10:55:26,431 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-04 10:55:26,431 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,431 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,433 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:26,433 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:26,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:26,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:26,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:26,433 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:26,433 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:26,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:26,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:26,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,433 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,433 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:26,434 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,434 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,434 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:26,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,434 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-04 10:55:26,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:26,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:26,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,434 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-04 10:55:26,434 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-04 10:55:26,434 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-04 10:55:26,434 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-04 10:55:26,434 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-04 10:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:26,435 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-04 10:55:26,435 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-04 10:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:55:26,435 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-04 10:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-04 10:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-04 10:55:26,435 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-04 10:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-04 10:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-04 10:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-04 10:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-04 10:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-04 10:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:26,435 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:26,435 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:26,435 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. started... 2023-05-04 10:55:26,436 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ae8b725404c1d45e9f47aaaa71304e53 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:26,436 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,436 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,437 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,437 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,437 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:26,437 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:26,444 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf1/7830076b2e0b46b2be3303a07917d6b1 2023-05-04 10:55:26,456 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf2/4443e12660814fe68debbf3b2d74ce1c 2023-05-04 10:55:26,460 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf1/7830076b2e0b46b2be3303a07917d6b1 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/7830076b2e0b46b2be3303a07917d6b1 2023-05-04 10:55:26,463 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/7830076b2e0b46b2be3303a07917d6b1, entries=10, sequenceid=71, filesize=5.0 K 2023-05-04 10:55:26,464 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf2/4443e12660814fe68debbf3b2d74ce1c as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/4443e12660814fe68debbf3b2d74ce1c 2023-05-04 10:55:26,467 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/4443e12660814fe68debbf3b2d74ce1c, entries=10, sequenceid=71, filesize=5.0 K 2023-05-04 10:55:26,468 INFO [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ae8b725404c1d45e9f47aaaa71304e53 in 33ms, sequenceid=71, compaction requested=true 2023-05-04 10:55:26,468 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ae8b725404c1d45e9f47aaaa71304e53: 2023-05-04 10:55:26,468 DEBUG [rs(jenkins-hbase4.apache.org,37343,1683197625438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:26,468 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:26,468 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:26,468 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,468 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-04 10:55:26,468 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-04 10:55:26,470 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,470 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:26,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:26,470 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,470 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:26,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:26,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:26,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:26,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:26,472 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-04 10:55:26,472 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4288448d[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:26,472 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-04 10:55:26,472 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,474 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,474 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,474 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,474 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:55:26,474 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-04 10:55:26,474 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,474 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:26,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,474 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:26,474 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:55:26,474 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-04 10:55:26,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,474 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:26,474 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-04 10:55:26,474 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-04 10:55:26,474 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:26,475 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-04 10:55:26,475 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:26,475 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-04 10:55:26,476 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-04 10:55:26,476 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,476 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-04 10:55:26,476 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:26,476 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:26,476 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-04 10:55:26,476 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-04 10:55:26,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:26,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:26,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:26,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:26,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:26,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:26,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:26,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,37343,1683197625438': 2023-05-04 10:55:26,479 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-04 10:55:26,479 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-04 10:55:26,479 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:26,479 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:26,479 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-04 10:55:26,479 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:26,481 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,481 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,481 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:26,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,481 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,481 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:26,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:26,481 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:26,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:26,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,481 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:26,481 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:26,482 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,482 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:26,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:26,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,484 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:26,484 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-04 10:55:26,484 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,491 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,491 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,491 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:26,491 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:26,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:26,491 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:26,491 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:26,491 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:26,491 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:26,491 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:26,491 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:26,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:26,491 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-04 10:55:26,492 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:26,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:26,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:26,491 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:26,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:26,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:26,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:26,491 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,492 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:26,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:26,492 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:26,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:26,492 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:26,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:26,492 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,492 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,492 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:26,492 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,492 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-04 10:55:26,592 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:26,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:26,852 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-05-04 10:55:26,853 DEBUG [Listener at localhost/35483] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:26,854 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:26,855 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:26,855 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:26,855 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:26,856 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:55:26,856 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:26,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:55:26,856 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:55:27,104 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,104 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:55:27,104 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:55:27,104 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:55:27,105 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-05-04 10:55:27,105 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:55:27,105 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:55:27,105 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:55:27,105 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:55:27,106 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:55:27,107 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-05-04 10:55:27,108 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:55:27,108 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:55:27,108 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:55:27,108 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:55:27,109 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:27,109 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:55:27,109 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:27,110 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:27,110 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:27,110 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:55:27,110 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:27,110 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:27,111 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:27,111 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:55:27,111 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:55:27,111 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:27,112 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:27,112 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:27,112 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0def560f25a82ca6fb786e8c5e288761, daughterA=f1f0e2a8ad3e3492a8c308585d3cf77b, daughterB=1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:55:27,112 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:27,113 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:27,113 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:27,113 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=6e4db4ef42259e1e3f31375f39c7adca, daughterA=4bb4acbdcedf2f8531e747ec59b78d24, daughterB=307b832391c13214298b00417783c5e0 2023-05-04 10:55:27,113 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:27,113 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:27,114 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:27,114 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:27,115 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42285,1683197625528, table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=9de9ca68de03518c48ec4415c14457cc, daughterA=248805fe1fdf27869faf29c5509ba1c6, daughterB=11f1194eaf50bbc8eb140bdfb097c7b7 2023-05-04 10:55:27,115 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:27,115 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:27,115 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:27,116 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:27,116 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:27,116 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:27,116 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,117 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:55:27,117 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,117 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,117 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,117 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-05-04 10:55:27,118 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:55:27,118 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:55:27,119 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197757119 2023-05-04 10:55:27,121 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:55:27,121 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,123 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:27,123 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ae8b725404c1d45e9f47aaaa71304e53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:27,124 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:55:27,124 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197787124 2023-05-04 10:55:27,124 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197847124 2023-05-04 10:55:27,124 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:55:27,124 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:55:27,126 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:55:27,126 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:55:27,127 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:55:27,374 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,374 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:55:27,374 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:55:27,374 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:55:27,374 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:55:27,375 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:55:27,375 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:55:27,375 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:55:27,375 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:55:27,375 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:55:27,377 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-05-04 10:55:27,378 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:55:27,378 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:55:27,378 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:55:27,378 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:55:27,378 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:27,378 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:55:27,379 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:27,379 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:27,379 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:27,379 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:55:27,379 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:27,379 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:27,379 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:27,380 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:55:27,380 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:55:27,380 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:27,380 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:27,381 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:27,381 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0def560f25a82ca6fb786e8c5e288761, daughterA=f1f0e2a8ad3e3492a8c308585d3cf77b, daughterB=1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:55:27,381 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:27,381 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:27,382 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:27,382 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=6e4db4ef42259e1e3f31375f39c7adca, daughterA=4bb4acbdcedf2f8531e747ec59b78d24, daughterB=307b832391c13214298b00417783c5e0 2023-05-04 10:55:27,382 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:27,382 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:27,382 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:27,382 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:27,382 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42285,1683197625528, table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=9de9ca68de03518c48ec4415c14457cc, daughterA=248805fe1fdf27869faf29c5509ba1c6, daughterB=11f1194eaf50bbc8eb140bdfb097c7b7 2023-05-04 10:55:27,383 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:27,383 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:27,383 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:27,383 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:27,383 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:27,384 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:27,384 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,384 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:55:27,384 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,384 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,384 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,385 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-05-04 10:55:27,385 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:55:27,386 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:55:27,386 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197757386 2023-05-04 10:55:27,388 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:55:27,388 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,391 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:27,391 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ae8b725404c1d45e9f47aaaa71304e53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:27,391 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:55:27,391 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197787391 2023-05-04 10:55:27,391 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197847391 2023-05-04 10:55:27,391 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-04 10:55:27,391 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:55:27,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN}] 2023-05-04 10:55:27,394 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:55:27,394 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:55:27,394 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:55:27,641 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,642 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:55:27,642 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:55:27,642 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:55:27,642 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:55:27,642 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:55:27,642 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:55:27,642 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:55:27,642 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:55:27,643 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:55:27,644 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-05-04 10:55:27,644 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:55:27,645 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:55:27,645 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:55:27,645 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:55:27,645 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:27,645 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:55:27,645 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:27,645 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:27,646 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:27,646 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:55:27,646 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:27,646 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:27,646 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:27,646 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:55:27,647 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:55:27,647 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:27,647 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:27,647 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:27,647 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0def560f25a82ca6fb786e8c5e288761, daughterA=f1f0e2a8ad3e3492a8c308585d3cf77b, daughterB=1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:55:27,647 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:27,647 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:27,648 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:27,648 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=6e4db4ef42259e1e3f31375f39c7adca, daughterA=4bb4acbdcedf2f8531e747ec59b78d24, daughterB=307b832391c13214298b00417783c5e0 2023-05-04 10:55:27,648 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:27,648 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:27,648 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:27,649 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:27,649 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42285,1683197625528, table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=9de9ca68de03518c48ec4415c14457cc, daughterA=248805fe1fdf27869faf29c5509ba1c6, daughterB=11f1194eaf50bbc8eb140bdfb097c7b7 2023-05-04 10:55:27,649 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:27,649 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:27,649 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:27,649 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:27,649 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:27,650 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:27,650 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,650 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:55:27,650 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,650 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,650 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,650 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-04 10:55:27,651 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:55:27,651 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:55:27,652 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197757652 2023-05-04 10:55:27,654 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,654 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:55:27,656 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:27,656 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ae8b725404c1d45e9f47aaaa71304e53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:27,657 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:55:27,657 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197787657 2023-05-04 10:55:27,657 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197847657 2023-05-04 10:55:27,657 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:55:27,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN}] 2023-05-04 10:55:27,657 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:55:27,660 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN 2023-05-04 10:55:27,661 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=ae8b725404c1d45e9f47aaaa71304e53, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:27,662 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197727661"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197727661"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197727661"}]},"ts":"1683197727661"} 2023-05-04 10:55:27,663 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438}] 2023-05-04 10:55:27,816 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:27,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ae8b725404c1d45e9f47aaaa71304e53, disabling compactions & flushes 2023-05-04 10:55:27,816 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:27,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:27,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. after waiting 0 ms 2023-05-04 10:55:27,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:27,816 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ae8b725404c1d45e9f47aaaa71304e53 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-04 10:55:27,825 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf1/596be24eb0c147328793596f1fbb4132 2023-05-04 10:55:27,836 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf2/5a019f2b0ce94e78931238293b677564 2023-05-04 10:55:27,840 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf1/596be24eb0c147328793596f1fbb4132 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/596be24eb0c147328793596f1fbb4132 2023-05-04 10:55:27,843 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/596be24eb0c147328793596f1fbb4132, entries=8, sequenceid=82, filesize=5.0 K 2023-05-04 10:55:27,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/.tmp/cf2/5a019f2b0ce94e78931238293b677564 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/5a019f2b0ce94e78931238293b677564 2023-05-04 10:55:27,848 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/5a019f2b0ce94e78931238293b677564, entries=8, sequenceid=82, filesize=5.0 K 2023-05-04 10:55:27,849 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ae8b725404c1d45e9f47aaaa71304e53 in 33ms, sequenceid=82, compaction requested=true 2023-05-04 10:55:27,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-04 10:55:27,853 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:55:27,854 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. 2023-05-04 10:55:27,854 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ae8b725404c1d45e9f47aaaa71304e53: 2023-05-04 10:55:27,856 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:27,856 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=ae8b725404c1d45e9f47aaaa71304e53, regionState=CLOSED 2023-05-04 10:55:27,857 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197727856"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197727856"}]},"ts":"1683197727856"} 2023-05-04 10:55:27,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-05-04 10:55:27,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438 in 195 msec 2023-05-04 10:55:27,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-05-04 10:55:27,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN in 202 msec 2023-05-04 10:55:27,863 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:55:27,863 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,863 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:55:27,863 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:55:27,907 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,907 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:55:27,907 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:55:27,907 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:55:27,907 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:55:27,908 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:55:27,908 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:55:27,908 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:55:27,908 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:55:27,908 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:55:27,909 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-04 10:55:27,910 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:55:27,910 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:55:27,910 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:55:27,910 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:55:27,910 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:55:27,910 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:55:27,911 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:27,911 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:55:27,911 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:27,911 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:27,911 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:27,911 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:55:27,911 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:27,912 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:27,912 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:27,912 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:55:27,912 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:55:27,912 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:27,912 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:27,913 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:27,913 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0def560f25a82ca6fb786e8c5e288761, daughterA=f1f0e2a8ad3e3492a8c308585d3cf77b, daughterB=1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:55:27,913 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:27,913 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:27,913 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:27,913 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=6e4db4ef42259e1e3f31375f39c7adca, daughterA=4bb4acbdcedf2f8531e747ec59b78d24, daughterB=307b832391c13214298b00417783c5e0 2023-05-04 10:55:27,913 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:27,913 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:27,914 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:27,914 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:27,914 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42285,1683197625528, table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=9de9ca68de03518c48ec4415c14457cc, daughterA=248805fe1fdf27869faf29c5509ba1c6, daughterB=11f1194eaf50bbc8eb140bdfb097c7b7 2023-05-04 10:55:27,914 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:27,914 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:27,915 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:27,915 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:27,915 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:27,915 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:27,916 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,916 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN 2023-05-04 10:55:27,916 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:27,916 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:55:27,916 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,916 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,916 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,916 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:27,916 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:27,916 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-04 10:55:27,917 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:55:27,917 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:55:27,918 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197757918 2023-05-04 10:55:27,919 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:55:27,919 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:27,922 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:27,922 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ae8b725404c1d45e9f47aaaa71304e53, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:27,922 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:55:27,922 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197787922 2023-05-04 10:55:27,922 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197847922 2023-05-04 10:55:27,922 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:55:27,924 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:55:27,933 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=ae8b725404c1d45e9f47aaaa71304e53, threads=14 2023-05-04 10:55:27,933 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/1f022c1ccf24485e99e47cc107060387 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:27,933 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/4443e12660814fe68debbf3b2d74ce1c for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:27,934 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/5a019f2b0ce94e78931238293b677564 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:27,934 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/724e838af6de4b788c2d4e879968d8df for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:27,934 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/659180a757a544d5858aec0baae07501 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:27,934 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/95c03e8f9e614998a9a57fa46315f3fe for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:27,934 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/ec68aac4dd8b426f9f4503314c0964be for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:27,934 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/17b163b7ed58427cb3d0eff92a1687e6 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:27,934 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/485bec54ae1147ff854248d9b1f4ce44 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:27,934 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/2ed8f38d4ec34201af96d9bd26a06832 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:27,934 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/494a5629975e4feb90318ed90245269b for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:27,935 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/7830076b2e0b46b2be3303a07917d6b1 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:27,935 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/596be24eb0c147328793596f1fbb4132 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:27,935 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/8b785c69c5ce402ea7e230aa15f2dd2b for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,122 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/659180a757a544d5858aec0baae07501 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,124 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/95c03e8f9e614998a9a57fa46315f3fe for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,124 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/8b785c69c5ce402ea7e230aa15f2dd2b for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,132 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/494a5629975e4feb90318ed90245269b for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,147 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/5a019f2b0ce94e78931238293b677564 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,147 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/485bec54ae1147ff854248d9b1f4ce44 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,152 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/2ed8f38d4ec34201af96d9bd26a06832 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,164 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/7830076b2e0b46b2be3303a07917d6b1 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,165 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/596be24eb0c147328793596f1fbb4132 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,167 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/4443e12660814fe68debbf3b2d74ce1c for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,176 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/ec68aac4dd8b426f9f4503314c0964be for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,182 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/1f022c1ccf24485e99e47cc107060387 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,191 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-04 10:55:28,508 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/17b163b7ed58427cb3d0eff92a1687e6 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,532 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/724e838af6de4b788c2d4e879968d8df for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,533 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region ae8b725404c1d45e9f47aaaa71304e53 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-04 10:55:28,563 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:55:28,563 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:28,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:55:28,563 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:55:28,672 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:28,673 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:55:28,673 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:55:28,673 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:55:28,673 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:55:28,673 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:55:28,673 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:55:28,673 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:55:28,673 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:55:28,674 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:55:28,675 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-05-04 10:55:28,675 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:55:28,675 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:55:28,676 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:55:28,676 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:55:28,676 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:55:28,676 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:55:28,676 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:28,677 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:55:28,677 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:28,677 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:28,677 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:28,677 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:55:28,678 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:28,678 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:28,678 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:28,678 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:55:28,678 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:55:28,678 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:28,678 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:28,679 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:28,679 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0def560f25a82ca6fb786e8c5e288761, daughterA=f1f0e2a8ad3e3492a8c308585d3cf77b, daughterB=1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:55:28,679 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:28,679 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:28,679 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:28,679 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=6e4db4ef42259e1e3f31375f39c7adca, daughterA=4bb4acbdcedf2f8531e747ec59b78d24, daughterB=307b832391c13214298b00417783c5e0 2023-05-04 10:55:28,679 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:28,680 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:28,680 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:28,680 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:28,680 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42285,1683197625528, table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=9de9ca68de03518c48ec4415c14457cc, daughterA=248805fe1fdf27869faf29c5509ba1c6, daughterB=11f1194eaf50bbc8eb140bdfb097c7b7 2023-05-04 10:55:28,680 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:28,680 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:28,681 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:28,681 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:28,681 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:28,681 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:28,681 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:28,681 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN 2023-05-04 10:55:28,681 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:28,681 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:55:28,681 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:28,681 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:28,682 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:28,682 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:28,682 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:28,682 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-04 10:55:28,682 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:55:28,683 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:55:28,684 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197758684 2023-05-04 10:55:28,685 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:55:28,685 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:28,687 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:28,688 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ae8b725404c1d45e9f47aaaa71304e53, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:28,688 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:55:28,688 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197788688 2023-05-04 10:55:28,688 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197848688 2023-05-04 10:55:28,688 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-04 10:55:28,699 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=ae8b725404c1d45e9f47aaaa71304e53, threads=14 2023-05-04 10:55:28,699 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/1f022c1ccf24485e99e47cc107060387 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,699 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/4443e12660814fe68debbf3b2d74ce1c for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,699 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/5a019f2b0ce94e78931238293b677564 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,699 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/659180a757a544d5858aec0baae07501 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,700 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/724e838af6de4b788c2d4e879968d8df for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,700 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/ec68aac4dd8b426f9f4503314c0964be for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,700 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/95c03e8f9e614998a9a57fa46315f3fe for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,704 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/17b163b7ed58427cb3d0eff92a1687e6 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,704 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/2ed8f38d4ec34201af96d9bd26a06832 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,704 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/485bec54ae1147ff854248d9b1f4ce44 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,704 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/494a5629975e4feb90318ed90245269b for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,704 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/596be24eb0c147328793596f1fbb4132 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,704 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/7830076b2e0b46b2be3303a07917d6b1 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,706 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/8b785c69c5ce402ea7e230aa15f2dd2b for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,862 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/95c03e8f9e614998a9a57fa46315f3fe for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,868 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/17b163b7ed58427cb3d0eff92a1687e6 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,870 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/ec68aac4dd8b426f9f4503314c0964be for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,870 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/659180a757a544d5858aec0baae07501 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,870 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/5a019f2b0ce94e78931238293b677564 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,877 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/2ed8f38d4ec34201af96d9bd26a06832 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,880 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/596be24eb0c147328793596f1fbb4132 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,896 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/724e838af6de4b788c2d4e879968d8df for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,906 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/4443e12660814fe68debbf3b2d74ce1c for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,906 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/8b785c69c5ce402ea7e230aa15f2dd2b for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,909 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/494a5629975e4feb90318ed90245269b for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,909 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/485bec54ae1147ff854248d9b1f4ce44 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,910 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/7830076b2e0b46b2be3303a07917d6b1 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,911 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/1f022c1ccf24485e99e47cc107060387 for region: ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:28,911 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region ae8b725404c1d45e9f47aaaa71304e53 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-04 10:55:28,942 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:55:28,948 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-04 10:55:28,950 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-04 10:55:28,950 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:55:28,950 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:28,950 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:55:28,950 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:55:29,188 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,189 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:55:29,189 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:55:29,189 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:55:29,189 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:55:29,189 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:55:29,189 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:55:29,189 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:55:29,189 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:55:29,190 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:55:29,191 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-05-04 10:55:29,191 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:55:29,191 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:55:29,191 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:55:29,192 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:55:29,192 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:55:29,192 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:55:29,192 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:29,192 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:55:29,192 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:29,193 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:29,193 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:29,193 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:55:29,193 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:29,193 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:29,193 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:29,193 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:55:29,194 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:55:29,194 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:29,194 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:29,194 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:29,194 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0def560f25a82ca6fb786e8c5e288761, daughterA=f1f0e2a8ad3e3492a8c308585d3cf77b, daughterB=1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:55:29,194 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:29,194 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:29,194 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:29,194 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=6e4db4ef42259e1e3f31375f39c7adca, daughterA=4bb4acbdcedf2f8531e747ec59b78d24, daughterB=307b832391c13214298b00417783c5e0 2023-05-04 10:55:29,195 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:29,195 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:29,195 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:29,195 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:29,195 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42285,1683197625528, table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=9de9ca68de03518c48ec4415c14457cc, daughterA=248805fe1fdf27869faf29c5509ba1c6, daughterB=11f1194eaf50bbc8eb140bdfb097c7b7 2023-05-04 10:55:29,195 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:29,195 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:29,195 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:29,196 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:29,196 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:29,196 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:29,196 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,196 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN 2023-05-04 10:55:29,196 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:29,196 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:55:29,196 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,196 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,197 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,197 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:29,197 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:29,197 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-04 10:55:29,197 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:55:29,198 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:55:29,198 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197759198 2023-05-04 10:55:29,200 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:55:29,200 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,202 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:29,202 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ae8b725404c1d45e9f47aaaa71304e53, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:29,202 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:55:29,202 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197789202 2023-05-04 10:55:29,202 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197849202 2023-05-04 10:55:29,203 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-04 10:55:29,205 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:55:29,207 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:55:29,207 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,207 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:55:29,207 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:55:29,453 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,453 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:55:29,453 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:55:29,453 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:55:29,453 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:55:29,453 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:55:29,453 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:55:29,453 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:55:29,454 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:55:29,454 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:55:29,456 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-05-04 10:55:29,456 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:55:29,456 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:55:29,456 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:55:29,457 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:55:29,457 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:55:29,457 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:55:29,458 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:29,458 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:55:29,458 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:29,458 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:29,458 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:29,459 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:55:29,459 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:29,459 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:29,459 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:29,459 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:55:29,460 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:55:29,460 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:29,460 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:29,460 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:29,460 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0def560f25a82ca6fb786e8c5e288761, daughterA=f1f0e2a8ad3e3492a8c308585d3cf77b, daughterB=1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:55:29,461 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:29,461 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:29,461 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:29,461 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=6e4db4ef42259e1e3f31375f39c7adca, daughterA=4bb4acbdcedf2f8531e747ec59b78d24, daughterB=307b832391c13214298b00417783c5e0 2023-05-04 10:55:29,461 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:29,462 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:29,462 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:29,462 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:29,462 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42285,1683197625528, table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=9de9ca68de03518c48ec4415c14457cc, daughterA=248805fe1fdf27869faf29c5509ba1c6, daughterB=11f1194eaf50bbc8eb140bdfb097c7b7 2023-05-04 10:55:29,463 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:29,463 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:29,463 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:29,463 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:29,463 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:29,464 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:29,464 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,464 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN 2023-05-04 10:55:29,464 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:29,464 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:55:29,465 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,465 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,465 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,465 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:29,465 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:29,465 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-05-04 10:55:29,466 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:55:29,466 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:55:29,471 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197759471 2023-05-04 10:55:29,473 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,473 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:55:29,476 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:29,476 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ae8b725404c1d45e9f47aaaa71304e53, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:29,476 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:55:29,476 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197789476 2023-05-04 10:55:29,477 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197849477 2023-05-04 10:55:29,477 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:55:29,477 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:55:29,479 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197729479"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1683197729479"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1683197729479"}]},"ts":"1683197729479"} 2023-05-04 10:55:29,479 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683197729479"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197729479"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197729479"}]},"ts":"1683197729479"} 2023-05-04 10:55:29,479 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683197729479"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197729479"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197729479"}]},"ts":"1683197729479"} 2023-05-04 10:55:29,481 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:55:29,481 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:55:29,481 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:55:29,727 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,727 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:55:29,727 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:55:29,727 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:55:29,727 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:55:29,727 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:55:29,727 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:55:29,727 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:55:29,728 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:55:29,728 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:55:29,729 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-05-04 10:55:29,729 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:55:29,730 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:55:29,730 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:55:29,730 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:55:29,730 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:55:29,730 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:55:29,730 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:29,731 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:55:29,731 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:29,731 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:29,731 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:29,731 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:55:29,731 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:29,731 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:29,732 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:29,732 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:55:29,733 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:55:29,733 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:29,733 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:29,733 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:29,733 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0def560f25a82ca6fb786e8c5e288761, daughterA=f1f0e2a8ad3e3492a8c308585d3cf77b, daughterB=1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:55:29,734 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:29,734 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:29,734 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:29,734 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=6e4db4ef42259e1e3f31375f39c7adca, daughterA=4bb4acbdcedf2f8531e747ec59b78d24, daughterB=307b832391c13214298b00417783c5e0 2023-05-04 10:55:29,734 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:29,735 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:29,735 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:29,735 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:29,736 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42285,1683197625528, table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=9de9ca68de03518c48ec4415c14457cc, daughterA=248805fe1fdf27869faf29c5509ba1c6, daughterB=11f1194eaf50bbc8eb140bdfb097c7b7 2023-05-04 10:55:29,736 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:29,736 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:29,736 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:29,736 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:29,736 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:29,737 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:29,737 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,737 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN 2023-05-04 10:55:29,737 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:29,737 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:55:29,737 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,737 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,737 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,737 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:29,737 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:29,737 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-05-04 10:55:29,738 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:55:29,739 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:55:29,739 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197759739 2023-05-04 10:55:29,741 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,741 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:55:29,743 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:29,744 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ae8b725404c1d45e9f47aaaa71304e53, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:29,744 INFO [Listener at localhost/35483] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,37343,1683197625438, table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:29,744 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=0ab6af3ef2cc6eb6a9787f09a4f7dab6, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-04 10:55:29,744 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=7e41c805a52b2b44d18a415646f1f511, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-04 10:55:29,744 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:55:29,745 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197789745 2023-05-04 10:55:29,745 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197849745 2023-05-04 10:55:29,745 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-04 10:55:29,745 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683197729745"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1683197729745"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1683197729745"}]},"ts":"1683197729745"} 2023-05-04 10:55:29,745 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683197729745"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197729745"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197729745"}]},"ts":"1683197729745"} 2023-05-04 10:55:29,745 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683197729745"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197729745"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197729745"}]},"ts":"1683197729745"} 2023-05-04 10:55:29,747 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:55:29,750 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:55:29,750 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,750 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:55:29,750 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:55:29,995 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:29,995 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:55:29,995 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:55:29,995 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:55:29,996 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:55:29,996 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:55:29,996 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:55:29,996 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:55:29,996 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:55:29,997 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:55:29,998 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-05-04 10:55:29,998 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:55:29,998 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:55:29,998 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:55:29,998 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:55:29,998 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:55:29,999 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:55:29,999 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:29,999 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:55:29,999 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:29,999 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:29,999 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:29,999 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:55:30,000 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:30,000 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:30,000 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:30,000 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:55:30,000 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:55:30,000 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:30,000 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:30,001 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:30,001 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0def560f25a82ca6fb786e8c5e288761, daughterA=f1f0e2a8ad3e3492a8c308585d3cf77b, daughterB=1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:55:30,001 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:30,001 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:30,001 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:30,001 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=6e4db4ef42259e1e3f31375f39c7adca, daughterA=4bb4acbdcedf2f8531e747ec59b78d24, daughterB=307b832391c13214298b00417783c5e0 2023-05-04 10:55:30,001 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:30,001 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:30,001 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:30,002 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:30,002 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42285,1683197625528, table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=9de9ca68de03518c48ec4415c14457cc, daughterA=248805fe1fdf27869faf29c5509ba1c6, daughterB=11f1194eaf50bbc8eb140bdfb097c7b7 2023-05-04 10:55:30,002 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:30,002 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:30,002 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:30,002 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:30,002 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:30,002 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:30,003 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,003 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN 2023-05-04 10:55:30,003 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:30,003 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:55:30,003 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,003 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,003 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,003 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:30,003 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:30,003 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-04 10:55:30,004 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:55:30,004 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:55:30,005 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197760005 2023-05-04 10:55:30,006 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:55:30,006 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,009 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:30,009 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ae8b725404c1d45e9f47aaaa71304e53, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:30,009 INFO [Listener at localhost/35483] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,37343,1683197625438, table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:30,009 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=0ab6af3ef2cc6eb6a9787f09a4f7dab6, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-04 10:55:30,009 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=7e41c805a52b2b44d18a415646f1f511, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-04 10:55:30,009 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:55:30,009 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197790009 2023-05-04 10:55:30,009 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197850009 2023-05-04 10:55:30,010 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:55:30,010 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:55:30,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0ab6af3ef2cc6eb6a9787f09a4f7dab6, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7e41c805a52b2b44d18a415646f1f511, ASSIGN}] 2023-05-04 10:55:30,012 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:55:30,012 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:55:30,012 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:55:30,091 ERROR [BP-2139253951-172.31.14.131-1683197620399 heartbeating to localhost/127.0.0.1:44245] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2139253951-172.31.14.131-1683197620399 (Datanode Uuid 9432e34f-f582-4227-a602-1b8f52f4fa10) service to localhost/127.0.0.1:44245 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-05-04 10:55:30,260 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,260 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:55:30,260 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:55:30,260 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:55:30,260 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:55:30,260 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:55:30,260 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:55:30,260 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:55:30,260 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:55:30,261 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:55:30,262 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-05-04 10:55:30,262 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:55:30,263 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:55:30,263 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:55:30,263 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:55:30,263 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:55:30,263 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:55:30,263 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:30,264 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:55:30,264 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:30,264 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:30,264 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:30,264 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:55:30,264 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:30,264 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:30,264 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:30,265 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:55:30,265 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:55:30,265 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:30,265 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:30,265 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:30,266 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0def560f25a82ca6fb786e8c5e288761, daughterA=f1f0e2a8ad3e3492a8c308585d3cf77b, daughterB=1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:55:30,266 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:30,266 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:30,266 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:30,266 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=6e4db4ef42259e1e3f31375f39c7adca, daughterA=4bb4acbdcedf2f8531e747ec59b78d24, daughterB=307b832391c13214298b00417783c5e0 2023-05-04 10:55:30,267 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:30,267 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:30,267 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:30,267 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:30,268 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42285,1683197625528, table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=9de9ca68de03518c48ec4415c14457cc, daughterA=248805fe1fdf27869faf29c5509ba1c6, daughterB=11f1194eaf50bbc8eb140bdfb097c7b7 2023-05-04 10:55:30,268 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:30,268 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:30,268 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:30,268 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:30,269 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:30,269 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:30,269 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,269 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN 2023-05-04 10:55:30,269 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:30,270 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:55:30,270 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,270 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,270 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,270 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:30,270 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:30,270 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-05-04 10:55:30,271 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:55:30,271 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:55:30,272 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197760272 2023-05-04 10:55:30,274 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,274 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:55:30,276 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:30,277 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ae8b725404c1d45e9f47aaaa71304e53, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:30,277 INFO [Listener at localhost/35483] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,37343,1683197625438, table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:30,277 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=0ab6af3ef2cc6eb6a9787f09a4f7dab6, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-04 10:55:30,277 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=7e41c805a52b2b44d18a415646f1f511, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-04 10:55:30,277 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:55:30,277 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197790277 2023-05-04 10:55:30,277 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197850277 2023-05-04 10:55:30,277 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-04 10:55:30,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0ab6af3ef2cc6eb6a9787f09a4f7dab6, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7e41c805a52b2b44d18a415646f1f511, ASSIGN}] 2023-05-04 10:55:30,278 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:55:30,280 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7e41c805a52b2b44d18a415646f1f511, ASSIGN 2023-05-04 10:55:30,280 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7e41c805a52b2b44d18a415646f1f511, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,37343,1683197625438; forceNewPlan=false, retain=false 2023-05-04 10:55:30,281 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0ab6af3ef2cc6eb6a9787f09a4f7dab6, ASSIGN 2023-05-04 10:55:30,281 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0ab6af3ef2cc6eb6a9787f09a4f7dab6, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,37343,1683197625438; forceNewPlan=false, retain=false 2023-05-04 10:55:30,430 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-04 10:55:30,432 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=7e41c805a52b2b44d18a415646f1f511, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:30,433 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683197730432"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197730432"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197730432"}]},"ts":"1683197730432"} 2023-05-04 10:55:30,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE; OpenRegionProcedure 7e41c805a52b2b44d18a415646f1f511, server=jenkins-hbase4.apache.org,37343,1683197625438}] 2023-05-04 10:55:30,436 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=0ab6af3ef2cc6eb6a9787f09a4f7dab6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:30,436 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683197730436"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197730436"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197730436"}]},"ts":"1683197730436"} 2023-05-04 10:55:30,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=115, state=RUNNABLE; OpenRegionProcedure 0ab6af3ef2cc6eb6a9787f09a4f7dab6, server=jenkins-hbase4.apache.org,37343,1683197625438}] 2023-05-04 10:55:30,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511. 2023-05-04 10:55:30,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7e41c805a52b2b44d18a415646f1f511, NAME => 'testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.', STARTKEY => '41', ENDKEY => ''} 2023-05-04 10:55:30,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:55:30,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:55:30,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,590 INFO [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,591 DEBUG [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1 2023-05-04 10:55:30,591 DEBUG [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1 2023-05-04 10:55:30,591 INFO [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7e41c805a52b2b44d18a415646f1f511 columnFamilyName cf1 2023-05-04 10:55:30,603 DEBUG [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/17b163b7ed58427cb3d0eff92a1687e6.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/17b163b7ed58427cb3d0eff92a1687e6-top 2023-05-04 10:55:30,606 DEBUG [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/2ed8f38d4ec34201af96d9bd26a06832.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/2ed8f38d4ec34201af96d9bd26a06832-top 2023-05-04 10:55:30,609 DEBUG [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/485bec54ae1147ff854248d9b1f4ce44.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/485bec54ae1147ff854248d9b1f4ce44-top 2023-05-04 10:55:30,612 DEBUG [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/494a5629975e4feb90318ed90245269b.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/494a5629975e4feb90318ed90245269b-top 2023-05-04 10:55:30,615 DEBUG [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/596be24eb0c147328793596f1fbb4132.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/596be24eb0c147328793596f1fbb4132-top 2023-05-04 10:55:30,619 DEBUG [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/7830076b2e0b46b2be3303a07917d6b1.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/7830076b2e0b46b2be3303a07917d6b1-top 2023-05-04 10:55:30,622 DEBUG [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/8b785c69c5ce402ea7e230aa15f2dd2b.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/8b785c69c5ce402ea7e230aa15f2dd2b-top 2023-05-04 10:55:30,623 INFO [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] regionserver.HStore(310): Store=7e41c805a52b2b44d18a415646f1f511/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:30,623 INFO [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,624 DEBUG [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2 2023-05-04 10:55:30,624 DEBUG [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2 2023-05-04 10:55:30,624 INFO [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7e41c805a52b2b44d18a415646f1f511 columnFamilyName cf2 2023-05-04 10:55:30,636 DEBUG [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/1f022c1ccf24485e99e47cc107060387.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/1f022c1ccf24485e99e47cc107060387-top 2023-05-04 10:55:30,640 DEBUG [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/4443e12660814fe68debbf3b2d74ce1c.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/4443e12660814fe68debbf3b2d74ce1c-top 2023-05-04 10:55:30,643 DEBUG [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/5a019f2b0ce94e78931238293b677564.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/5a019f2b0ce94e78931238293b677564-top 2023-05-04 10:55:30,647 DEBUG [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/659180a757a544d5858aec0baae07501.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/659180a757a544d5858aec0baae07501-top 2023-05-04 10:55:30,650 DEBUG [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/724e838af6de4b788c2d4e879968d8df.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/724e838af6de4b788c2d4e879968d8df-top 2023-05-04 10:55:30,653 DEBUG [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/95c03e8f9e614998a9a57fa46315f3fe.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/95c03e8f9e614998a9a57fa46315f3fe-top 2023-05-04 10:55:30,657 DEBUG [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/ec68aac4dd8b426f9f4503314c0964be.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/ec68aac4dd8b426f9f4503314c0964be-top 2023-05-04 10:55:30,657 INFO [StoreOpener-7e41c805a52b2b44d18a415646f1f511-1] regionserver.HStore(310): Store=7e41c805a52b2b44d18a415646f1f511/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:30,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,659 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:55:30,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7e41c805a52b2b44d18a415646f1f511; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72325480, jitterRate=0.07773363590240479}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:55:30,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7e41c805a52b2b44d18a415646f1f511: 2023-05-04 10:55:30,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511., pid=117, masterSystemTime=1683197730585 2023-05-04 10:55:30,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:55:30,664 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:55:30,665 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:55:30,665 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:55:30,665 INFO [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511. 2023-05-04 10:55:30,665 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HStore(1912): 7e41c805a52b2b44d18a415646f1f511/cf1 is initiating minor compaction (all files) 2023-05-04 10:55:30,665 INFO [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511. 2023-05-04 10:55:30,665 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HStore(1912): 7e41c805a52b2b44d18a415646f1f511/cf2 is initiating minor compaction (all files) 2023-05-04 10:55:30,665 INFO [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 7e41c805a52b2b44d18a415646f1f511/cf1 in testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511. 2023-05-04 10:55:30,665 INFO [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 7e41c805a52b2b44d18a415646f1f511/cf2 in testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511. 2023-05-04 10:55:30,665 INFO [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/8b785c69c5ce402ea7e230aa15f2dd2b.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/8b785c69c5ce402ea7e230aa15f2dd2b-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/485bec54ae1147ff854248d9b1f4ce44.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/485bec54ae1147ff854248d9b1f4ce44-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/494a5629975e4feb90318ed90245269b.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/494a5629975e4feb90318ed90245269b-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/17b163b7ed58427cb3d0eff92a1687e6.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/17b163b7ed58427cb3d0eff92a1687e6-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/2ed8f38d4ec34201af96d9bd26a06832.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/2ed8f38d4ec34201af96d9bd26a06832-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/7830076b2e0b46b2be3303a07917d6b1.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/7830076b2e0b46b2be3303a07917d6b1-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/596be24eb0c147328793596f1fbb4132.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/596be24eb0c147328793596f1fbb4132-top] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/.tmp, totalSize=34.9 K 2023-05-04 10:55:30,665 INFO [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/659180a757a544d5858aec0baae07501.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/659180a757a544d5858aec0baae07501-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/ec68aac4dd8b426f9f4503314c0964be.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/ec68aac4dd8b426f9f4503314c0964be-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/1f022c1ccf24485e99e47cc107060387.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/1f022c1ccf24485e99e47cc107060387-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/724e838af6de4b788c2d4e879968d8df.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/724e838af6de4b788c2d4e879968d8df-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/95c03e8f9e614998a9a57fa46315f3fe.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/95c03e8f9e614998a9a57fa46315f3fe-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/4443e12660814fe68debbf3b2d74ce1c.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/4443e12660814fe68debbf3b2d74ce1c-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/5a019f2b0ce94e78931238293b677564.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/5a019f2b0ce94e78931238293b677564-top] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/.tmp, totalSize=34.9 K 2023-05-04 10:55:30,666 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting 8b785c69c5ce402ea7e230aa15f2dd2b.ae8b725404c1d45e9f47aaaa71304e53, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683197725529 2023-05-04 10:55:30,666 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting 659180a757a544d5858aec0baae07501.ae8b725404c1d45e9f47aaaa71304e53, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683197725529 2023-05-04 10:55:30,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511. 2023-05-04 10:55:30,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511. 2023-05-04 10:55:30,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6. 2023-05-04 10:55:30,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0ab6af3ef2cc6eb6a9787f09a4f7dab6, NAME => 'testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.', STARTKEY => '', ENDKEY => '41'} 2023-05-04 10:55:30,666 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting 485bec54ae1147ff854248d9b1f4ce44.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683197725705 2023-05-04 10:55:30,666 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting ec68aac4dd8b426f9f4503314c0964be.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683197725705 2023-05-04 10:55:30,666 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=7e41c805a52b2b44d18a415646f1f511, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:30,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:55:30,667 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683197730666"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197730666"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197730666"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197730666"}]},"ts":"1683197730666"} 2023-05-04 10:55:30,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 0ab6af3ef2cc6eb6a9787f09a4f7dab6 2023-05-04 10:55:30,667 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting 494a5629975e4feb90318ed90245269b.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683197725880 2023-05-04 10:55:30,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:55:30,667 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting 1f022c1ccf24485e99e47cc107060387.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683197725880 2023-05-04 10:55:30,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0ab6af3ef2cc6eb6a9787f09a4f7dab6 2023-05-04 10:55:30,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0ab6af3ef2cc6eb6a9787f09a4f7dab6 2023-05-04 10:55:30,667 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting 17b163b7ed58427cb3d0eff92a1687e6.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683197726066 2023-05-04 10:55:30,668 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting 724e838af6de4b788c2d4e879968d8df.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683197726066 2023-05-04 10:55:30,668 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting 2ed8f38d4ec34201af96d9bd26a06832.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1683197726245 2023-05-04 10:55:30,668 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting 95c03e8f9e614998a9a57fa46315f3fe.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1683197726245 2023-05-04 10:55:30,668 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting 7830076b2e0b46b2be3303a07917d6b1.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1683197726418 2023-05-04 10:55:30,669 INFO [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0ab6af3ef2cc6eb6a9787f09a4f7dab6 2023-05-04 10:55:30,669 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting 4443e12660814fe68debbf3b2d74ce1c.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1683197726418 2023-05-04 10:55:30,669 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-05-04 10:55:30,669 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting 5a019f2b0ce94e78931238293b677564.ae8b725404c1d45e9f47aaaa71304e53, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683197726594 2023-05-04 10:55:30,669 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 7e41c805a52b2b44d18a415646f1f511, server=jenkins-hbase4.apache.org,37343,1683197625438 in 234 msec 2023-05-04 10:55:30,670 DEBUG [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1 2023-05-04 10:55:30,670 DEBUG [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1 2023-05-04 10:55:30,670 INFO [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0ab6af3ef2cc6eb6a9787f09a4f7dab6 columnFamilyName cf1 2023-05-04 10:55:30,670 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7e41c805a52b2b44d18a415646f1f511, ASSIGN in 392 msec 2023-05-04 10:55:30,672 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting 596be24eb0c147328793596f1fbb4132.ae8b725404c1d45e9f47aaaa71304e53, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683197726594 2023-05-04 10:55:30,682 INFO [RS:0;jenkins-hbase4:37343-longCompactions-0] throttle.PressureAwareThroughputController(145): 7e41c805a52b2b44d18a415646f1f511#cf2#compaction#108 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:55:30,685 DEBUG [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/17b163b7ed58427cb3d0eff92a1687e6.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/17b163b7ed58427cb3d0eff92a1687e6-bottom 2023-05-04 10:55:30,688 DEBUG [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/2ed8f38d4ec34201af96d9bd26a06832.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/2ed8f38d4ec34201af96d9bd26a06832-bottom 2023-05-04 10:55:30,693 INFO [RS:0;jenkins-hbase4:37343-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7e41c805a52b2b44d18a415646f1f511#cf1#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:55:30,709 DEBUG [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/485bec54ae1147ff854248d9b1f4ce44.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/485bec54ae1147ff854248d9b1f4ce44-bottom 2023-05-04 10:55:30,713 DEBUG [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/494a5629975e4feb90318ed90245269b.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/494a5629975e4feb90318ed90245269b-bottom 2023-05-04 10:55:30,721 DEBUG [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/596be24eb0c147328793596f1fbb4132.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/596be24eb0c147328793596f1fbb4132-bottom 2023-05-04 10:55:30,721 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/.tmp/cf2/b8a8af74fb6e4afebf5c89586a2ee22f as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/b8a8af74fb6e4afebf5c89586a2ee22f 2023-05-04 10:55:30,729 DEBUG [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/7830076b2e0b46b2be3303a07917d6b1.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/7830076b2e0b46b2be3303a07917d6b1-bottom 2023-05-04 10:55:30,731 INFO [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7e41c805a52b2b44d18a415646f1f511/cf2 of 7e41c805a52b2b44d18a415646f1f511 into b8a8af74fb6e4afebf5c89586a2ee22f(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:55:30,731 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7e41c805a52b2b44d18a415646f1f511: 2023-05-04 10:55:30,731 INFO [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511., storeName=7e41c805a52b2b44d18a415646f1f511/cf2, priority=9, startTime=1683197730664; duration=0sec 2023-05-04 10:55:30,731 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/.tmp/cf1/2523212dc13e4843aebf54e3340915d8 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/2523212dc13e4843aebf54e3340915d8 2023-05-04 10:55:30,731 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:55:30,733 DEBUG [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/8b785c69c5ce402ea7e230aa15f2dd2b.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/8b785c69c5ce402ea7e230aa15f2dd2b-bottom 2023-05-04 10:55:30,734 INFO [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] regionserver.HStore(310): Store=0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:30,734 INFO [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0ab6af3ef2cc6eb6a9787f09a4f7dab6 2023-05-04 10:55:30,735 DEBUG [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2 2023-05-04 10:55:30,735 DEBUG [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2 2023-05-04 10:55:30,736 INFO [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0ab6af3ef2cc6eb6a9787f09a4f7dab6 columnFamilyName cf2 2023-05-04 10:55:30,736 INFO [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7e41c805a52b2b44d18a415646f1f511/cf1 of 7e41c805a52b2b44d18a415646f1f511 into 2523212dc13e4843aebf54e3340915d8(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:55:30,736 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7e41c805a52b2b44d18a415646f1f511: 2023-05-04 10:55:30,736 INFO [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511., storeName=7e41c805a52b2b44d18a415646f1f511/cf1, priority=9, startTime=1683197730663; duration=0sec 2023-05-04 10:55:30,736 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:55:30,749 DEBUG [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/1f022c1ccf24485e99e47cc107060387.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/1f022c1ccf24485e99e47cc107060387-bottom 2023-05-04 10:55:30,754 DEBUG [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/4443e12660814fe68debbf3b2d74ce1c.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/4443e12660814fe68debbf3b2d74ce1c-bottom 2023-05-04 10:55:30,757 DEBUG [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/5a019f2b0ce94e78931238293b677564.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/5a019f2b0ce94e78931238293b677564-bottom 2023-05-04 10:55:30,760 DEBUG [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/659180a757a544d5858aec0baae07501.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/659180a757a544d5858aec0baae07501-bottom 2023-05-04 10:55:30,763 DEBUG [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/724e838af6de4b788c2d4e879968d8df.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/724e838af6de4b788c2d4e879968d8df-bottom 2023-05-04 10:55:30,767 DEBUG [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/95c03e8f9e614998a9a57fa46315f3fe.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/95c03e8f9e614998a9a57fa46315f3fe-bottom 2023-05-04 10:55:30,771 DEBUG [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/ec68aac4dd8b426f9f4503314c0964be.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/ec68aac4dd8b426f9f4503314c0964be-bottom 2023-05-04 10:55:30,771 INFO [StoreOpener-0ab6af3ef2cc6eb6a9787f09a4f7dab6-1] regionserver.HStore(310): Store=0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:30,772 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6 2023-05-04 10:55:30,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6 2023-05-04 10:55:30,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:55:30,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0ab6af3ef2cc6eb6a9787f09a4f7dab6 2023-05-04 10:55:30,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0ab6af3ef2cc6eb6a9787f09a4f7dab6; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68737652, jitterRate=0.024270832538604736}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:55:30,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0ab6af3ef2cc6eb6a9787f09a4f7dab6: 2023-05-04 10:55:30,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6., pid=118, masterSystemTime=1683197730585 2023-05-04 10:55:30,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:55:30,778 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:55:30,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-04 10:55:30,778 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:55:30,778 INFO [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6. 2023-05-04 10:55:30,778 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HStore(1912): 0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1 is initiating minor compaction (all files) 2023-05-04 10:55:30,779 INFO [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6. 2023-05-04 10:55:30,779 INFO [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1 in testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6. 2023-05-04 10:55:30,779 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HStore(1912): 0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2 is initiating minor compaction (all files) 2023-05-04 10:55:30,779 INFO [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2 in testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6. 2023-05-04 10:55:30,779 INFO [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/8b785c69c5ce402ea7e230aa15f2dd2b.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/8b785c69c5ce402ea7e230aa15f2dd2b-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/485bec54ae1147ff854248d9b1f4ce44.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/485bec54ae1147ff854248d9b1f4ce44-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/494a5629975e4feb90318ed90245269b.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/494a5629975e4feb90318ed90245269b-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/17b163b7ed58427cb3d0eff92a1687e6.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/17b163b7ed58427cb3d0eff92a1687e6-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/2ed8f38d4ec34201af96d9bd26a06832.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/2ed8f38d4ec34201af96d9bd26a06832-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/7830076b2e0b46b2be3303a07917d6b1.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/7830076b2e0b46b2be3303a07917d6b1-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/596be24eb0c147328793596f1fbb4132.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/596be24eb0c147328793596f1fbb4132-bottom] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/.tmp, totalSize=34.9 K 2023-05-04 10:55:30,779 INFO [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/659180a757a544d5858aec0baae07501.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/659180a757a544d5858aec0baae07501-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/ec68aac4dd8b426f9f4503314c0964be.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/ec68aac4dd8b426f9f4503314c0964be-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/1f022c1ccf24485e99e47cc107060387.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/1f022c1ccf24485e99e47cc107060387-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/724e838af6de4b788c2d4e879968d8df.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/724e838af6de4b788c2d4e879968d8df-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/95c03e8f9e614998a9a57fa46315f3fe.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/95c03e8f9e614998a9a57fa46315f3fe-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/4443e12660814fe68debbf3b2d74ce1c.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/4443e12660814fe68debbf3b2d74ce1c-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/5a019f2b0ce94e78931238293b677564.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/5a019f2b0ce94e78931238293b677564-bottom] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/.tmp, totalSize=34.9 K 2023-05-04 10:55:30,780 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting 8b785c69c5ce402ea7e230aa15f2dd2b.ae8b725404c1d45e9f47aaaa71304e53, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683197725529 2023-05-04 10:55:30,781 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting 659180a757a544d5858aec0baae07501.ae8b725404c1d45e9f47aaaa71304e53, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683197725529 2023-05-04 10:55:30,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6. 2023-05-04 10:55:30,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6. 2023-05-04 10:55:30,781 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting 485bec54ae1147ff854248d9b1f4ce44.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683197725705 2023-05-04 10:55:30,781 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting ec68aac4dd8b426f9f4503314c0964be.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683197725705 2023-05-04 10:55:30,782 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=0ab6af3ef2cc6eb6a9787f09a4f7dab6, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:30,782 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683197730781"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197730781"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197730781"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197730781"}]},"ts":"1683197730781"} 2023-05-04 10:55:30,782 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting 494a5629975e4feb90318ed90245269b.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683197725880 2023-05-04 10:55:30,782 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting 1f022c1ccf24485e99e47cc107060387.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683197725880 2023-05-04 10:55:30,783 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting 17b163b7ed58427cb3d0eff92a1687e6.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683197726066 2023-05-04 10:55:30,783 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting 724e838af6de4b788c2d4e879968d8df.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683197726066 2023-05-04 10:55:30,783 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting 2ed8f38d4ec34201af96d9bd26a06832.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683197726245 2023-05-04 10:55:30,784 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting 7830076b2e0b46b2be3303a07917d6b1.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683197726418 2023-05-04 10:55:30,784 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting 95c03e8f9e614998a9a57fa46315f3fe.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683197726245 2023-05-04 10:55:30,784 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] compactions.Compactor(207): Compacting 596be24eb0c147328793596f1fbb4132.ae8b725404c1d45e9f47aaaa71304e53, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683197726594 2023-05-04 10:55:30,784 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting 4443e12660814fe68debbf3b2d74ce1c.ae8b725404c1d45e9f47aaaa71304e53, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683197726418 2023-05-04 10:55:30,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=115 2023-05-04 10:55:30,785 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] compactions.Compactor(207): Compacting 5a019f2b0ce94e78931238293b677564.ae8b725404c1d45e9f47aaaa71304e53, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683197726594 2023-05-04 10:55:30,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 0ab6af3ef2cc6eb6a9787f09a4f7dab6, server=jenkins-hbase4.apache.org,37343,1683197625438 in 346 msec 2023-05-04 10:55:30,787 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-05-04 10:55:30,787 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0ab6af3ef2cc6eb6a9787f09a4f7dab6, ASSIGN in 508 msec 2023-05-04 10:55:30,787 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-04 10:55:30,787 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:30,787 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:55:30,787 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-04 10:55:30,796 INFO [RS:0;jenkins-hbase4:37343-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0ab6af3ef2cc6eb6a9787f09a4f7dab6#cf1#compaction#110 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:55:30,796 INFO [RS:0;jenkins-hbase4:37343-longCompactions-0] throttle.PressureAwareThroughputController(145): 0ab6af3ef2cc6eb6a9787f09a4f7dab6#cf2#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:55:30,813 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/.tmp/cf2/dac884629b824d09bab9d227a04ad7d3 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/dac884629b824d09bab9d227a04ad7d3 2023-05-04 10:55:30,814 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/.tmp/cf1/ec0ab23cdc93431ca7e742f03ad1fc87 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/ec0ab23cdc93431ca7e742f03ad1fc87 2023-05-04 10:55:30,818 INFO [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1 of 0ab6af3ef2cc6eb6a9787f09a4f7dab6 into ec0ab23cdc93431ca7e742f03ad1fc87(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:55:30,818 INFO [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2 of 0ab6af3ef2cc6eb6a9787f09a4f7dab6 into dac884629b824d09bab9d227a04ad7d3(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:55:30,818 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0ab6af3ef2cc6eb6a9787f09a4f7dab6: 2023-05-04 10:55:30,818 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0ab6af3ef2cc6eb6a9787f09a4f7dab6: 2023-05-04 10:55:30,818 INFO [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6., storeName=0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1, priority=9, startTime=1683197730777; duration=0sec 2023-05-04 10:55:30,818 INFO [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6., storeName=0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2, priority=9, startTime=1683197730778; duration=0sec 2023-05-04 10:55:30,818 DEBUG [RS:0;jenkins-hbase4:37343-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:55:30,818 DEBUG [RS:0;jenkins-hbase4:37343-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:55:31,028 INFO [Listener at localhost/35483] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:31,029 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-04 10:55:31,029 INFO [Listener at localhost/35483] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:55:31,029 INFO [Listener at localhost/35483] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:55:31,029 ERROR [Listener at localhost/35483] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-04 10:55:31,029 INFO [Listener at localhost/35483] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-04 10:55:31,029 INFO [Listener at localhost/35483] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-04 10:55:31,029 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-04 10:55:31,029 INFO [Listener at localhost/35483] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-04 10:55:31,030 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-04 10:55:31,031 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-05-04 10:55:31,032 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-04 10:55:31,032 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-04 10:55:31,032 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 18, 23] 2023-05-04 10:55:31,032 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-05-04 10:55:31,032 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 16, 21] 2023-05-04 10:55:31,032 DEBUG [Listener at localhost/35483] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-05-04 10:55:31,032 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-04 10:55:31,033 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-04 10:55:31,033 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-04 10:55:31,033 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-04 10:55:31,033 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:31,034 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f9694df3407e6abdcc846e20c3b7c433, daughterA=068a8cdea69c89e269dfa78e5a33d56b, daughterB=3c5731995a4a3f9265c35d3aa9f41462 2023-05-04 10:55:31,034 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:31,034 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-04 10:55:31,034 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:31,034 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=fdafca090ed7cc34c4ad878aac762e9e, daughterA=4110ba1eeda845d33e575cf303f91707, daughterB=9db21ffe8cb46d9683ca276a900101ec 2023-05-04 10:55:31,035 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:31,035 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-04 10:55:31,035 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:31,035 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ff484ddaba859c44b9223e0332036369, daughterA=e232f81670423fc5259ac5116e147909, daughterB=40856f69942775d59f85b65348effee4 2023-05-04 10:55:31,035 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ff484ddaba859c44b9223e0332036369, ASSIGN 2023-05-04 10:55:31,035 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:31,036 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-04 10:55:31,036 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:31,036 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0def560f25a82ca6fb786e8c5e288761, daughterA=f1f0e2a8ad3e3492a8c308585d3cf77b, daughterB=1ba1c1861bf6bdd2c2ab43bc2bebbd9a 2023-05-04 10:55:31,036 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:31,036 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-04 10:55:31,037 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:31,037 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=6e4db4ef42259e1e3f31375f39c7adca, daughterA=4bb4acbdcedf2f8531e747ec59b78d24, daughterB=307b832391c13214298b00417783c5e0 2023-05-04 10:55:31,037 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:31,037 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-04 10:55:31,037 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:31,038 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:31,038 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,42285,1683197625528, table=testRollbackForSplitTableRegionWithReplica, region=afb93548d078e4e75c09efee8b4edf98 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=9de9ca68de03518c48ec4415c14457cc, daughterA=248805fe1fdf27869faf29c5509ba1c6, daughterB=11f1194eaf50bbc8eb140bdfb097c7b7 2023-05-04 10:55:31,038 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:31,038 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-04 10:55:31,038 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:31,038 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:31,039 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-04 10:55:31,039 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:31,040 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:31,040 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN 2023-05-04 10:55:31,040 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:31,040 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0ab6af3ef2cc6eb6a9787f09a4f7dab6, ASSIGN 2023-05-04 10:55:31,040 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7e41c805a52b2b44d18a415646f1f511, ASSIGN 2023-05-04 10:55:31,041 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 7e41c805a52b2b44d18a415646f1f511, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:31,041 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 0ab6af3ef2cc6eb6a9787f09a4f7dab6, server=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:31,041 DEBUG [Listener at localhost/35483] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 held the lock before restarting, call acquireLock to restore it. 2023-05-04 10:55:31,041 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:31,041 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:31,041 INFO [Listener at localhost/35483] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:31,041 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:31,041 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ae8b725404c1d45e9f47aaaa71304e53, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:31,041 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0ab6af3ef2cc6eb6a9787f09a4f7dab6, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:31,041 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7e41c805a52b2b44d18a415646f1f511, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:31,041 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 7e41c805a52b2b44d18a415646f1f511, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:31,041 DEBUG [Listener at localhost/35483] procedure2.Procedure(946): pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 0ab6af3ef2cc6eb6a9787f09a4f7dab6, server=jenkins-hbase4.apache.org,37343,1683197625438 didn't hold the lock before restarting, skip acquiring lock. 2023-05-04 10:55:31,042 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-05-04 10:55:31,042 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-04 10:55:31,043 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528, table=hbase:meta, region=1588230740 2023-05-04 10:55:31,045 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683197761045 2023-05-04 10:55:31,048 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:31,048 DEBUG [Listener at localhost/35483] assignment.AssignmentManager(1517): Joining cluster... 2023-05-04 10:55:31,051 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=468c84c2202e377e79810bff249e3ae7, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:31,051 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=ae8b725404c1d45e9f47aaaa71304e53, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=2 2023-05-04 10:55:31,052 INFO [Listener at localhost/35483] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,37343,1683197625438, table=testRecoveryAndDoubleExecution, region=ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,052 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=0ab6af3ef2cc6eb6a9787f09a4f7dab6, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=86 2023-05-04 10:55:31,052 DEBUG [Listener at localhost/35483] assignment.RegionStateStore(147): Load hbase:meta entry region=7e41c805a52b2b44d18a415646f1f511, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37343,1683197625438, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438, openSeqNum=86 2023-05-04 10:55:31,052 INFO [Listener at localhost/35483] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-04 10:55:31,052 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683197791052 2023-05-04 10:55:31,052 INFO [Listener at localhost/35483] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683197851052 2023-05-04 10:55:31,052 INFO [Listener at localhost/35483] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-04 10:55:31,053 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-04 10:55:31,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ae8b725404c1d45e9f47aaaa71304e53, daughterA=0ab6af3ef2cc6eb6a9787f09a4f7dab6, daughterB=7e41c805a52b2b44d18a415646f1f511 in 4.2000 sec 2023-05-04 10:55:31,091 ERROR [BP-2139253951-172.31.14.131-1683197620399 heartbeating to localhost/127.0.0.1:44245] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2139253951-172.31.14.131-1683197620399 (Datanode Uuid 4e8a05ef-1582-45ea-8d3b-86bbf8f33f91) service to localhost/127.0.0.1:44245 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-05-04 10:55:31,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37343] ipc.CallRunner(144): callId: 886 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:39976 deadline: 1683197791304, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53. is not online on jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:31,415 DEBUG [Listener at localhost/35483] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-04 10:55:31,423 INFO [Listener at localhost/35483] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-05-04 10:55:31,425 DEBUG [Listener at localhost/35483] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-04 10:55:31,433 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:55:31,435 INFO [Listener at localhost/35483] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-05-04 10:55:31,435 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-05-04 10:55:31,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:31,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-05-04 10:55:31,439 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197731438"}]},"ts":"1683197731438"} 2023-05-04 10:55:31,439 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-05-04 10:55:31,441 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-05-04 10:55:31,441 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0ab6af3ef2cc6eb6a9787f09a4f7dab6, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7e41c805a52b2b44d18a415646f1f511, UNASSIGN}] 2023-05-04 10:55:31,442 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7e41c805a52b2b44d18a415646f1f511, UNASSIGN 2023-05-04 10:55:31,442 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=7e41c805a52b2b44d18a415646f1f511, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:31,443 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683197731442"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197731442"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197731442"}]},"ts":"1683197731442"} 2023-05-04 10:55:31,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 7e41c805a52b2b44d18a415646f1f511, server=jenkins-hbase4.apache.org,37343,1683197625438}] 2023-05-04 10:55:31,445 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0ab6af3ef2cc6eb6a9787f09a4f7dab6, UNASSIGN 2023-05-04 10:55:31,446 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=0ab6af3ef2cc6eb6a9787f09a4f7dab6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:31,446 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683197731446"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197731446"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197731446"}]},"ts":"1683197731446"} 2023-05-04 10:55:31,447 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 0ab6af3ef2cc6eb6a9787f09a4f7dab6, server=jenkins-hbase4.apache.org,37343,1683197625438}] 2023-05-04 10:55:31,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-05-04 10:55:31,595 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:31,595 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7e41c805a52b2b44d18a415646f1f511, disabling compactions & flushes 2023-05-04 10:55:31,595 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511. 2023-05-04 10:55:31,595 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511. 2023-05-04 10:55:31,595 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511. after waiting 0 ms 2023-05-04 10:55:31,595 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511. 2023-05-04 10:55:31,596 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/8b785c69c5ce402ea7e230aa15f2dd2b.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/8b785c69c5ce402ea7e230aa15f2dd2b-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/485bec54ae1147ff854248d9b1f4ce44.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/485bec54ae1147ff854248d9b1f4ce44-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/494a5629975e4feb90318ed90245269b.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/494a5629975e4feb90318ed90245269b-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/17b163b7ed58427cb3d0eff92a1687e6.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/17b163b7ed58427cb3d0eff92a1687e6-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/2ed8f38d4ec34201af96d9bd26a06832.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/2ed8f38d4ec34201af96d9bd26a06832-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/7830076b2e0b46b2be3303a07917d6b1.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/7830076b2e0b46b2be3303a07917d6b1-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/596be24eb0c147328793596f1fbb4132.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/596be24eb0c147328793596f1fbb4132-top] to archive 2023-05-04 10:55:31,597 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:55:31,599 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/8b785c69c5ce402ea7e230aa15f2dd2b.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/8b785c69c5ce402ea7e230aa15f2dd2b.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,600 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/485bec54ae1147ff854248d9b1f4ce44.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/485bec54ae1147ff854248d9b1f4ce44.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,601 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/494a5629975e4feb90318ed90245269b.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/494a5629975e4feb90318ed90245269b.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,602 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/17b163b7ed58427cb3d0eff92a1687e6.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/17b163b7ed58427cb3d0eff92a1687e6.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,603 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/2ed8f38d4ec34201af96d9bd26a06832.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/2ed8f38d4ec34201af96d9bd26a06832.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,604 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/7830076b2e0b46b2be3303a07917d6b1.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/7830076b2e0b46b2be3303a07917d6b1.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,606 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/596be24eb0c147328793596f1fbb4132.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/596be24eb0c147328793596f1fbb4132.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,607 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/659180a757a544d5858aec0baae07501.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/659180a757a544d5858aec0baae07501-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/ec68aac4dd8b426f9f4503314c0964be.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/ec68aac4dd8b426f9f4503314c0964be-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/1f022c1ccf24485e99e47cc107060387.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/1f022c1ccf24485e99e47cc107060387-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/724e838af6de4b788c2d4e879968d8df.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/724e838af6de4b788c2d4e879968d8df-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/95c03e8f9e614998a9a57fa46315f3fe.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/95c03e8f9e614998a9a57fa46315f3fe-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/4443e12660814fe68debbf3b2d74ce1c.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/4443e12660814fe68debbf3b2d74ce1c-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/5a019f2b0ce94e78931238293b677564.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/5a019f2b0ce94e78931238293b677564-top] to archive 2023-05-04 10:55:31,608 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:55:31,609 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/659180a757a544d5858aec0baae07501.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/659180a757a544d5858aec0baae07501.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,611 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/ec68aac4dd8b426f9f4503314c0964be.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/ec68aac4dd8b426f9f4503314c0964be.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,612 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/1f022c1ccf24485e99e47cc107060387.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/1f022c1ccf24485e99e47cc107060387.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,613 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/724e838af6de4b788c2d4e879968d8df.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/724e838af6de4b788c2d4e879968d8df.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,614 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/95c03e8f9e614998a9a57fa46315f3fe.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/95c03e8f9e614998a9a57fa46315f3fe.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,615 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/4443e12660814fe68debbf3b2d74ce1c.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/4443e12660814fe68debbf3b2d74ce1c.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,616 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/5a019f2b0ce94e78931238293b677564.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/5a019f2b0ce94e78931238293b677564.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,620 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-04 10:55:31,620 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:55:31,620 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511. 2023-05-04 10:55:31,620 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7e41c805a52b2b44d18a415646f1f511: 2023-05-04 10:55:31,623 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:31,623 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 0ab6af3ef2cc6eb6a9787f09a4f7dab6 2023-05-04 10:55:31,623 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0ab6af3ef2cc6eb6a9787f09a4f7dab6, disabling compactions & flushes 2023-05-04 10:55:31,623 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6. 2023-05-04 10:55:31,623 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6. 2023-05-04 10:55:31,623 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6. after waiting 0 ms 2023-05-04 10:55:31,623 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6. 2023-05-04 10:55:31,623 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=7e41c805a52b2b44d18a415646f1f511, regionState=CLOSED 2023-05-04 10:55:31,623 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683197731623"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197731623"}]},"ts":"1683197731623"} 2023-05-04 10:55:31,625 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/8b785c69c5ce402ea7e230aa15f2dd2b.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/8b785c69c5ce402ea7e230aa15f2dd2b-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/485bec54ae1147ff854248d9b1f4ce44.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/485bec54ae1147ff854248d9b1f4ce44-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/494a5629975e4feb90318ed90245269b.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/494a5629975e4feb90318ed90245269b-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/17b163b7ed58427cb3d0eff92a1687e6.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/17b163b7ed58427cb3d0eff92a1687e6-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/2ed8f38d4ec34201af96d9bd26a06832.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/2ed8f38d4ec34201af96d9bd26a06832-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/7830076b2e0b46b2be3303a07917d6b1.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/7830076b2e0b46b2be3303a07917d6b1-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/596be24eb0c147328793596f1fbb4132.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/596be24eb0c147328793596f1fbb4132-bottom] to archive 2023-05-04 10:55:31,625 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:55:31,626 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-05-04 10:55:31,626 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 7e41c805a52b2b44d18a415646f1f511, server=jenkins-hbase4.apache.org,37343,1683197625438 in 182 msec 2023-05-04 10:55:31,627 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/8b785c69c5ce402ea7e230aa15f2dd2b.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/8b785c69c5ce402ea7e230aa15f2dd2b.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,628 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7e41c805a52b2b44d18a415646f1f511, UNASSIGN in 185 msec 2023-05-04 10:55:31,628 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/485bec54ae1147ff854248d9b1f4ce44.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/485bec54ae1147ff854248d9b1f4ce44.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,629 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/494a5629975e4feb90318ed90245269b.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/494a5629975e4feb90318ed90245269b.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,630 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/17b163b7ed58427cb3d0eff92a1687e6.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/17b163b7ed58427cb3d0eff92a1687e6.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,631 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/2ed8f38d4ec34201af96d9bd26a06832.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/2ed8f38d4ec34201af96d9bd26a06832.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,632 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/7830076b2e0b46b2be3303a07917d6b1.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/7830076b2e0b46b2be3303a07917d6b1.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,633 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/596be24eb0c147328793596f1fbb4132.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/596be24eb0c147328793596f1fbb4132.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,635 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/659180a757a544d5858aec0baae07501.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/659180a757a544d5858aec0baae07501-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/ec68aac4dd8b426f9f4503314c0964be.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/ec68aac4dd8b426f9f4503314c0964be-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/1f022c1ccf24485e99e47cc107060387.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/1f022c1ccf24485e99e47cc107060387-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/724e838af6de4b788c2d4e879968d8df.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/724e838af6de4b788c2d4e879968d8df-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/95c03e8f9e614998a9a57fa46315f3fe.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/95c03e8f9e614998a9a57fa46315f3fe-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/4443e12660814fe68debbf3b2d74ce1c.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/4443e12660814fe68debbf3b2d74ce1c-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/5a019f2b0ce94e78931238293b677564.ae8b725404c1d45e9f47aaaa71304e53->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/5a019f2b0ce94e78931238293b677564-bottom] to archive 2023-05-04 10:55:31,636 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:55:31,637 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/659180a757a544d5858aec0baae07501.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/659180a757a544d5858aec0baae07501.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,638 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/ec68aac4dd8b426f9f4503314c0964be.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/ec68aac4dd8b426f9f4503314c0964be.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,639 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/1f022c1ccf24485e99e47cc107060387.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/1f022c1ccf24485e99e47cc107060387.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,640 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/724e838af6de4b788c2d4e879968d8df.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/724e838af6de4b788c2d4e879968d8df.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,641 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/95c03e8f9e614998a9a57fa46315f3fe.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/95c03e8f9e614998a9a57fa46315f3fe.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,642 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/4443e12660814fe68debbf3b2d74ce1c.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/4443e12660814fe68debbf3b2d74ce1c.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,643 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/5a019f2b0ce94e78931238293b677564.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/5a019f2b0ce94e78931238293b677564.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,646 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-04 10:55:31,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:55:31,647 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6. 2023-05-04 10:55:31,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0ab6af3ef2cc6eb6a9787f09a4f7dab6: 2023-05-04 10:55:31,649 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 0ab6af3ef2cc6eb6a9787f09a4f7dab6 2023-05-04 10:55:31,650 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=0ab6af3ef2cc6eb6a9787f09a4f7dab6, regionState=CLOSED 2023-05-04 10:55:31,650 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683197731650"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197731650"}]},"ts":"1683197731650"} 2023-05-04 10:55:31,652 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-05-04 10:55:31,653 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 0ab6af3ef2cc6eb6a9787f09a4f7dab6, server=jenkins-hbase4.apache.org,37343,1683197625438 in 204 msec 2023-05-04 10:55:31,654 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-05-04 10:55:31,654 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0ab6af3ef2cc6eb6a9787f09a4f7dab6, UNASSIGN in 212 msec 2023-05-04 10:55:31,654 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197731654"}]},"ts":"1683197731654"} 2023-05-04 10:55:31,655 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-05-04 10:55:31,657 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-05-04 10:55:31,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 222 msec 2023-05-04 10:55:31,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-05-04 10:55:31,741 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-05-04 10:55:31,741 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-05-04 10:55:31,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:31,744 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:31,745 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:31,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-05-04 10:55:31,749 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,749 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6 2023-05-04 10:55:31,749 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:31,751 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/recovered.edits] 2023-05-04 10:55:31,751 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/recovered.edits] 2023-05-04 10:55:31,752 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/recovered.edits] 2023-05-04 10:55:31,755 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/17b163b7ed58427cb3d0eff92a1687e6 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/17b163b7ed58427cb3d0eff92a1687e6 2023-05-04 10:55:31,755 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/.regioninfo to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/.regioninfo 2023-05-04 10:55:31,755 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/.regioninfo to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/.regioninfo 2023-05-04 10:55:31,757 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/2ed8f38d4ec34201af96d9bd26a06832 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/2ed8f38d4ec34201af96d9bd26a06832 2023-05-04 10:55:31,759 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/485bec54ae1147ff854248d9b1f4ce44 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/485bec54ae1147ff854248d9b1f4ce44 2023-05-04 10:55:31,759 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf1/17b163b7ed58427cb3d0eff92a1687e6.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf1/17b163b7ed58427cb3d0eff92a1687e6.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,759 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/17b163b7ed58427cb3d0eff92a1687e6.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/17b163b7ed58427cb3d0eff92a1687e6.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,760 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/494a5629975e4feb90318ed90245269b to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/494a5629975e4feb90318ed90245269b 2023-05-04 10:55:31,760 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/2ed8f38d4ec34201af96d9bd26a06832.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/2ed8f38d4ec34201af96d9bd26a06832.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,760 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf1/2ed8f38d4ec34201af96d9bd26a06832.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf1/2ed8f38d4ec34201af96d9bd26a06832.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,763 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/596be24eb0c147328793596f1fbb4132 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/596be24eb0c147328793596f1fbb4132 2023-05-04 10:55:31,763 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/485bec54ae1147ff854248d9b1f4ce44.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/485bec54ae1147ff854248d9b1f4ce44.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,763 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf1/485bec54ae1147ff854248d9b1f4ce44.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf1/485bec54ae1147ff854248d9b1f4ce44.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,764 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/7830076b2e0b46b2be3303a07917d6b1 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/7830076b2e0b46b2be3303a07917d6b1 2023-05-04 10:55:31,764 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/494a5629975e4feb90318ed90245269b.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/494a5629975e4feb90318ed90245269b.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,765 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf1/494a5629975e4feb90318ed90245269b.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf1/494a5629975e4feb90318ed90245269b.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,766 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/8b785c69c5ce402ea7e230aa15f2dd2b to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf1/8b785c69c5ce402ea7e230aa15f2dd2b 2023-05-04 10:55:31,766 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/596be24eb0c147328793596f1fbb4132.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/596be24eb0c147328793596f1fbb4132.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,766 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf1/596be24eb0c147328793596f1fbb4132.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf1/596be24eb0c147328793596f1fbb4132.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,768 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/7830076b2e0b46b2be3303a07917d6b1.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/7830076b2e0b46b2be3303a07917d6b1.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,769 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf1/7830076b2e0b46b2be3303a07917d6b1.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf1/7830076b2e0b46b2be3303a07917d6b1.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,770 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/8b785c69c5ce402ea7e230aa15f2dd2b.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/8b785c69c5ce402ea7e230aa15f2dd2b.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,770 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/1f022c1ccf24485e99e47cc107060387 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/1f022c1ccf24485e99e47cc107060387 2023-05-04 10:55:31,770 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf1/8b785c69c5ce402ea7e230aa15f2dd2b.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf1/8b785c69c5ce402ea7e230aa15f2dd2b.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,771 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/4443e12660814fe68debbf3b2d74ce1c to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/4443e12660814fe68debbf3b2d74ce1c 2023-05-04 10:55:31,772 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/5a019f2b0ce94e78931238293b677564 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/5a019f2b0ce94e78931238293b677564 2023-05-04 10:55:31,773 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/1f022c1ccf24485e99e47cc107060387.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/1f022c1ccf24485e99e47cc107060387.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,774 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf2/1f022c1ccf24485e99e47cc107060387.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf2/1f022c1ccf24485e99e47cc107060387.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,774 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/659180a757a544d5858aec0baae07501 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/659180a757a544d5858aec0baae07501 2023-05-04 10:55:31,775 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/4443e12660814fe68debbf3b2d74ce1c.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/4443e12660814fe68debbf3b2d74ce1c.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,776 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf2/4443e12660814fe68debbf3b2d74ce1c.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf2/4443e12660814fe68debbf3b2d74ce1c.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,776 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/724e838af6de4b788c2d4e879968d8df to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/724e838af6de4b788c2d4e879968d8df 2023-05-04 10:55:31,777 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/5a019f2b0ce94e78931238293b677564.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/5a019f2b0ce94e78931238293b677564.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,778 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf2/5a019f2b0ce94e78931238293b677564.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf2/5a019f2b0ce94e78931238293b677564.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,778 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/95c03e8f9e614998a9a57fa46315f3fe to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/95c03e8f9e614998a9a57fa46315f3fe 2023-05-04 10:55:31,778 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/659180a757a544d5858aec0baae07501.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/659180a757a544d5858aec0baae07501.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,779 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf2/659180a757a544d5858aec0baae07501.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf2/659180a757a544d5858aec0baae07501.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,780 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/ec68aac4dd8b426f9f4503314c0964be to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/cf2/ec68aac4dd8b426f9f4503314c0964be 2023-05-04 10:55:31,781 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/724e838af6de4b788c2d4e879968d8df.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/724e838af6de4b788c2d4e879968d8df.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,781 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf2/724e838af6de4b788c2d4e879968d8df.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf2/724e838af6de4b788c2d4e879968d8df.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,782 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/95c03e8f9e614998a9a57fa46315f3fe.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/95c03e8f9e614998a9a57fa46315f3fe.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,783 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf2/95c03e8f9e614998a9a57fa46315f3fe.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf2/95c03e8f9e614998a9a57fa46315f3fe.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,784 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/recovered.edits/85.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53/recovered.edits/85.seqid 2023-05-04 10:55:31,784 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf2/ec68aac4dd8b426f9f4503314c0964be.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/7e41c805a52b2b44d18a415646f1f511/cf2/ec68aac4dd8b426f9f4503314c0964be.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,784 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/ec68aac4dd8b426f9f4503314c0964be.ae8b725404c1d45e9f47aaaa71304e53 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/ec68aac4dd8b426f9f4503314c0964be.ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,785 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/ae8b725404c1d45e9f47aaaa71304e53 2023-05-04 10:55:31,787 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/2523212dc13e4843aebf54e3340915d8 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf1/2523212dc13e4843aebf54e3340915d8 2023-05-04 10:55:31,787 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/ec0ab23cdc93431ca7e742f03ad1fc87 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf1/ec0ab23cdc93431ca7e742f03ad1fc87 2023-05-04 10:55:31,790 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/b8a8af74fb6e4afebf5c89586a2ee22f to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/cf2/b8a8af74fb6e4afebf5c89586a2ee22f 2023-05-04 10:55:31,790 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/dac884629b824d09bab9d227a04ad7d3 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/cf2/dac884629b824d09bab9d227a04ad7d3 2023-05-04 10:55:31,793 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/recovered.edits/90.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511/recovered.edits/90.seqid 2023-05-04 10:55:31,793 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/recovered.edits/90.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6/recovered.edits/90.seqid 2023-05-04 10:55:31,793 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/7e41c805a52b2b44d18a415646f1f511 2023-05-04 10:55:31,795 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testRecoveryAndDoubleExecution/0ab6af3ef2cc6eb6a9787f09a4f7dab6 2023-05-04 10:55:31,795 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-05-04 10:55:31,797 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:31,799 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-05-04 10:55:31,800 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-05-04 10:55:31,801 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:31,801 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-05-04 10:55:31,802 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197731801"}]},"ts":"9223372036854775807"} 2023-05-04 10:55:31,802 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197731801"}]},"ts":"9223372036854775807"} 2023-05-04 10:55:31,802 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197731801"}]},"ts":"9223372036854775807"} 2023-05-04 10:55:31,803 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-04 10:55:31,803 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ae8b725404c1d45e9f47aaaa71304e53, NAME => 'testRecoveryAndDoubleExecution,,1683197725026.ae8b725404c1d45e9f47aaaa71304e53.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 0ab6af3ef2cc6eb6a9787f09a4f7dab6, NAME => 'testRecoveryAndDoubleExecution,,1683197726853.0ab6af3ef2cc6eb6a9787f09a4f7dab6.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 7e41c805a52b2b44d18a415646f1f511, NAME => 'testRecoveryAndDoubleExecution,41,1683197726853.7e41c805a52b2b44d18a415646f1f511.', STARTKEY => '41', ENDKEY => ''}] 2023-05-04 10:55:31,803 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-05-04 10:55:31,803 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683197731803"}]},"ts":"9223372036854775807"} 2023-05-04 10:55:31,805 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-05-04 10:55:31,806 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-04 10:55:31,807 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 65 msec 2023-05-04 10:55:31,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-05-04 10:55:31,846 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-05-04 10:55:31,862 INFO [Listener at localhost/35483] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=375 (was 369) - Thread LEAK? -, OpenFileDescriptor=829 (was 742) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=175 (was 181), ProcessCount=170 (was 170), AvailableMemoryMB=6313 (was 6327) 2023-05-04 10:55:31,876 INFO [Listener at localhost/35483] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=375, OpenFileDescriptor=829, MaxFileDescriptor=60000, SystemLoadAverage=175, ProcessCount=170, AvailableMemoryMB=6312 2023-05-04 10:55:31,877 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:55:31,879 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-04 10:55:31,880 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-04 10:55:31,881 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-04 10:55:31,882 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-05-04 10:55:31,883 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-04 10:55:31,885 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:31,885 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d empty. 2023-05-04 10:55:31,885 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:31,886 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-05-04 10:55:31,901 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-05-04 10:55:31,902 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6ad719aaf381b8be3588a9ac72a2d00d, NAME => 'testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp 2023-05-04 10:55:31,912 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:55:31,913 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1604): Closing 6ad719aaf381b8be3588a9ac72a2d00d, disabling compactions & flushes 2023-05-04 10:55:31,913 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:31,913 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:31,913 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. after waiting 0 ms 2023-05-04 10:55:31,913 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:31,913 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:31,913 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 6ad719aaf381b8be3588a9ac72a2d00d: 2023-05-04 10:55:31,915 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-05-04 10:55:31,916 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683197731915"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197731915"}]},"ts":"1683197731915"} 2023-05-04 10:55:31,917 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-04 10:55:31,917 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-04 10:55:31,917 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197731917"}]},"ts":"1683197731917"} 2023-05-04 10:55:31,918 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-05-04 10:55:31,921 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-04 10:55:31,922 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-04 10:55:31,922 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-04 10:55:31,922 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-04 10:55:31,922 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-04 10:55:31,922 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6ad719aaf381b8be3588a9ac72a2d00d, ASSIGN}] 2023-05-04 10:55:31,923 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6ad719aaf381b8be3588a9ac72a2d00d, ASSIGN 2023-05-04 10:55:31,923 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6ad719aaf381b8be3588a9ac72a2d00d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42285,1683197625528; forceNewPlan=false, retain=false 2023-05-04 10:55:32,074 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-04 10:55:32,075 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=6ad719aaf381b8be3588a9ac72a2d00d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:32,075 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683197732075"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197732075"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197732075"}]},"ts":"1683197732075"} 2023-05-04 10:55:32,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 6ad719aaf381b8be3588a9ac72a2d00d, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:55:32,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:32,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6ad719aaf381b8be3588a9ac72a2d00d, NAME => 'testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d.', STARTKEY => '', ENDKEY => ''} 2023-05-04 10:55:32,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:55:32,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:32,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:55:32,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:32,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:32,238 INFO [StoreOpener-6ad719aaf381b8be3588a9ac72a2d00d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:32,240 DEBUG [StoreOpener-6ad719aaf381b8be3588a9ac72a2d00d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1 2023-05-04 10:55:32,240 DEBUG [StoreOpener-6ad719aaf381b8be3588a9ac72a2d00d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1 2023-05-04 10:55:32,240 INFO [StoreOpener-6ad719aaf381b8be3588a9ac72a2d00d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ad719aaf381b8be3588a9ac72a2d00d columnFamilyName cf1 2023-05-04 10:55:32,240 INFO [StoreOpener-6ad719aaf381b8be3588a9ac72a2d00d-1] regionserver.HStore(310): Store=6ad719aaf381b8be3588a9ac72a2d00d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:32,241 INFO [StoreOpener-6ad719aaf381b8be3588a9ac72a2d00d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:32,242 DEBUG [StoreOpener-6ad719aaf381b8be3588a9ac72a2d00d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2 2023-05-04 10:55:32,242 DEBUG [StoreOpener-6ad719aaf381b8be3588a9ac72a2d00d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2 2023-05-04 10:55:32,242 INFO [StoreOpener-6ad719aaf381b8be3588a9ac72a2d00d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ad719aaf381b8be3588a9ac72a2d00d columnFamilyName cf2 2023-05-04 10:55:32,242 INFO [StoreOpener-6ad719aaf381b8be3588a9ac72a2d00d-1] regionserver.HStore(310): Store=6ad719aaf381b8be3588a9ac72a2d00d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:32,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:32,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:32,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:55:32,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:32,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-04 10:55:32,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6ad719aaf381b8be3588a9ac72a2d00d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59191470, jitterRate=-0.11797836422920227}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:55:32,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6ad719aaf381b8be3588a9ac72a2d00d: 2023-05-04 10:55:32,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d., pid=127, masterSystemTime=1683197732228 2023-05-04 10:55:32,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:32,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:32,250 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=6ad719aaf381b8be3588a9ac72a2d00d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:32,250 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683197732250"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197732250"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197732250"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197732250"}]},"ts":"1683197732250"} 2023-05-04 10:55:32,253 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-05-04 10:55:32,253 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 6ad719aaf381b8be3588a9ac72a2d00d, server=jenkins-hbase4.apache.org,42285,1683197625528 in 175 msec 2023-05-04 10:55:32,254 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-05-04 10:55:32,254 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6ad719aaf381b8be3588a9ac72a2d00d, ASSIGN in 331 msec 2023-05-04 10:55:32,255 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-04 10:55:32,255 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197732255"}]},"ts":"1683197732255"} 2023-05-04 10:55:32,256 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-05-04 10:55:32,258 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-05-04 10:55:32,259 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 378 msec 2023-05-04 10:55:32,388 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:32,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,390 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-04 10:55:32,390 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:32,390 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-04 10:55:32,390 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-04 10:55:32,391 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,391 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,393 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:32,393 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:32,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:32,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:32,393 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:32,393 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:32,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:32,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:32,394 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:32,394 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:32,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:32,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:32,394 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:32,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,394 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,394 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,394 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:32,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,395 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,395 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,395 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-04 10:55:32,395 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:32,395 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-04 10:55:32,395 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:32,395 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:32,395 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:32,395 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:32,395 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-04 10:55:32,395 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. started... 2023-05-04 10:55:32,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,395 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:32,395 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,396 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-04 10:55:32,397 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-04 10:55:32,396 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6ad719aaf381b8be3588a9ac72a2d00d 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-04 10:55:32,397 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-04 10:55:32,397 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,397 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-04 10:55:32,397 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:32,397 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-04 10:55:32,397 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-04 10:55:32,397 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-04 10:55:32,398 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,399 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,399 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,399 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:32,399 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:32,399 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:32,806 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf1/d82bab81d0784a8ea602e56ce8a8e720 2023-05-04 10:55:33,218 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf2/5c04bdb3b4d946ba9e732986652c7b10 2023-05-04 10:55:33,222 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf1/d82bab81d0784a8ea602e56ce8a8e720 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/d82bab81d0784a8ea602e56ce8a8e720 2023-05-04 10:55:33,225 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/d82bab81d0784a8ea602e56ce8a8e720, entries=2, sequenceid=6, filesize=4.8 K 2023-05-04 10:55:33,226 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf2/5c04bdb3b4d946ba9e732986652c7b10 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/5c04bdb3b4d946ba9e732986652c7b10 2023-05-04 10:55:33,229 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/5c04bdb3b4d946ba9e732986652c7b10, entries=2, sequenceid=6, filesize=4.8 K 2023-05-04 10:55:33,230 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 6ad719aaf381b8be3588a9ac72a2d00d in 834ms, sequenceid=6, compaction requested=false 2023-05-04 10:55:33,230 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-05-04 10:55:33,230 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6ad719aaf381b8be3588a9ac72a2d00d: 2023-05-04 10:55:33,230 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:33,230 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:33,231 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:33,231 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,231 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-04 10:55:33,231 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-04 10:55:33,232 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,232 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:33,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:33,232 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,232 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:33,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:33,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:33,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:33,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-04 10:55:33,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ee3f51e[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:33,234 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-04 10:55:33,234 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,236 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,236 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,236 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,236 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-04 10:55:33,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,237 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-04 10:55:33,237 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-04 10:55:33,237 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-04 10:55:33,237 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-04 10:55:33,236 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-04 10:55:33,237 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:33,236 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,237 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:33,237 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:33,237 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:33,238 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-04 10:55:33,238 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,238 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-04 10:55:33,238 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-04 10:55:33,238 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,238 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,238 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-04 10:55:33,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:33,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:33,238 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-04 10:55:33,239 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,239 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-04 10:55:33,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:33,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:33,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:33,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:55:33,241 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:55:33,241 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-04 10:55:33,241 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:33,241 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:33,241 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,241 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:33,243 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,243 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,243 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,243 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,243 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,243 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:33,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:33,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,243 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,243 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:33,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,244 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:33,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,244 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:33,246 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,246 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,246 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,253 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,253 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,253 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,253 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,253 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,253 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,253 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,253 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-04 10:55:33,253 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:33,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,253 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,253 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,253 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,253 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,253 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:33,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,254 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:33,253 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,254 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,254 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,254 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,254 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,254 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,254 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,354 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:33,355 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-04 10:55:33,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:33,366 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:33,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:33,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,368 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-04 10:55:33,368 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:33,368 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-04 10:55:33,368 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-04 10:55:33,368 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,368 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,369 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,369 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,369 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,369 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,370 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,370 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:33,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,370 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,370 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,370 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:33,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,371 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,371 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-04 10:55:33,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:33,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,371 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,371 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,371 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-04 10:55:33,371 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:33,371 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-04 10:55:33,371 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-04 10:55:33,371 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-04 10:55:33,371 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-04 10:55:33,371 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,371 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-04 10:55:33,371 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:33,371 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-04 10:55:33,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:33,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,371 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,372 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:33,372 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:33,372 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:33,372 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. started... 2023-05-04 10:55:33,372 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6ad719aaf381b8be3588a9ac72a2d00d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:33,373 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,373 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,373 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:33,376 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-04 10:55:33,376 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,376 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-04 10:55:33,376 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:33,376 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-04 10:55:33,376 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-04 10:55:33,376 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-04 10:55:33,378 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,378 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,378 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:33,389 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf1/85ba15d7901e41b0813ccf7e359d41ff 2023-05-04 10:55:33,412 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf2/8ad09df92c1d4d9f8445ce3e1c47713d 2023-05-04 10:55:33,417 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf1/85ba15d7901e41b0813ccf7e359d41ff as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/85ba15d7901e41b0813ccf7e359d41ff 2023-05-04 10:55:33,421 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/85ba15d7901e41b0813ccf7e359d41ff, entries=10, sequenceid=19, filesize=5.0 K 2023-05-04 10:55:33,422 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf2/8ad09df92c1d4d9f8445ce3e1c47713d as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/8ad09df92c1d4d9f8445ce3e1c47713d 2023-05-04 10:55:33,425 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/8ad09df92c1d4d9f8445ce3e1c47713d, entries=10, sequenceid=19, filesize=5.0 K 2023-05-04 10:55:33,426 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6ad719aaf381b8be3588a9ac72a2d00d in 54ms, sequenceid=19, compaction requested=false 2023-05-04 10:55:33,427 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6ad719aaf381b8be3588a9ac72a2d00d: 2023-05-04 10:55:33,427 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:33,427 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:33,427 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:33,427 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,427 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-04 10:55:33,427 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-04 10:55:33,429 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,429 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:33,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:33,430 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,430 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:33,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:33,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:33,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:33,432 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-04 10:55:33,432 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@373c0da4[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:33,432 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-04 10:55:33,432 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,433 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,433 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,433 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,434 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,434 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:33,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,434 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-04 10:55:33,434 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-04 10:55:33,434 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-04 10:55:33,434 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-04 10:55:33,434 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-04 10:55:33,434 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:33,434 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:33,434 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-04 10:55:33,434 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:33,435 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-04 10:55:33,435 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-04 10:55:33,435 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,435 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:33,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:33,435 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-04 10:55:33,436 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,436 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-04 10:55:33,435 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,436 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-04 10:55:33,436 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-04 10:55:33,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:33,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:33,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:33,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:55:33,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:55:33,440 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-04 10:55:33,440 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:33,440 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:33,440 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,440 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:33,441 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,441 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:33,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:33,441 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,441 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,441 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,441 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,442 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:33,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,441 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,442 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:33,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,443 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:33,444 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,445 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,445 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,453 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,453 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,453 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,453 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,453 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,453 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,453 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,453 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,453 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-04 10:55:33,453 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:33,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,453 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,453 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,453 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,454 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,454 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:33,454 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,454 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,454 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:33,454 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,454 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,454 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,454 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,554 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:33,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:33,566 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:33,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:33,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,568 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-04 10:55:33,568 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:33,568 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-04 10:55:33,568 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-04 10:55:33,568 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,568 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,570 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,570 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,570 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,570 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,570 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,570 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:33,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,571 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,571 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,571 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:33,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:33,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:33,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,571 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,571 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,571 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,571 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-04 10:55:33,571 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,572 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-04 10:55:33,572 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:33,572 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-04 10:55:33,572 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-04 10:55:33,572 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-04 10:55:33,573 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,573 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,573 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:33,579 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-04 10:55:33,579 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,579 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-04 10:55:33,579 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-04 10:55:33,579 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,579 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:33,579 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-04 10:55:33,579 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:33,579 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-04 10:55:33,579 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-04 10:55:33,579 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-04 10:55:33,579 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-04 10:55:33,579 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:33,579 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:33,579 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:33,579 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. started... 2023-05-04 10:55:33,579 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6ad719aaf381b8be3588a9ac72a2d00d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:33,580 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,580 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,580 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:33,594 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf1/cace4ee073a54636a9ade830c70d1e3e 2023-05-04 10:55:33,605 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf2/e794281b4afe4306b7cdf8935162548f 2023-05-04 10:55:33,608 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf1/cace4ee073a54636a9ade830c70d1e3e as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/cace4ee073a54636a9ade830c70d1e3e 2023-05-04 10:55:33,611 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/cace4ee073a54636a9ade830c70d1e3e, entries=10, sequenceid=32, filesize=5.0 K 2023-05-04 10:55:33,612 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf2/e794281b4afe4306b7cdf8935162548f as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/e794281b4afe4306b7cdf8935162548f 2023-05-04 10:55:33,615 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/e794281b4afe4306b7cdf8935162548f, entries=10, sequenceid=32, filesize=5.0 K 2023-05-04 10:55:33,616 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6ad719aaf381b8be3588a9ac72a2d00d in 37ms, sequenceid=32, compaction requested=true 2023-05-04 10:55:33,616 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6ad719aaf381b8be3588a9ac72a2d00d: 2023-05-04 10:55:33,616 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:33,616 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:33,616 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:33,616 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,616 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-04 10:55:33,616 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-04 10:55:33,618 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,618 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:33,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:33,618 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,618 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:33,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:33,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:33,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:33,620 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-04 10:55:33,620 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@33d8c90[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:33,620 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-04 10:55:33,620 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,621 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,621 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,621 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,621 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-04 10:55:33,622 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-04 10:55:33,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,622 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:33,622 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,622 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:33,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,622 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-04 10:55:33,622 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-04 10:55:33,622 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-04 10:55:33,622 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:33,622 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-04 10:55:33,622 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:33,625 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-04 10:55:33,625 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,625 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-04 10:55:33,626 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-04 10:55:33,626 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,627 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-04 10:55:33,627 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-04 10:55:33,627 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,627 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,627 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-04 10:55:33,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:33,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:33,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:33,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:33,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:33,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:55:33,630 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:55:33,630 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-04 10:55:33,630 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:33,630 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:33,630 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,630 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:33,631 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,631 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:33,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:33,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,631 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,631 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,631 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,632 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:33,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,632 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,632 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,632 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:33,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,633 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:33,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,634 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,634 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,635 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,637 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,637 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,637 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,637 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,637 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-04 10:55:33,637 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:33,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,637 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,637 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,637 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,638 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,638 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,638 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,637 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,638 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:33,638 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,638 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:33,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,638 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,638 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,638 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,638 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,638 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,638 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,738 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:33,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:33,750 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:33,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:33,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,751 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-04 10:55:33,751 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:33,751 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-04 10:55:33,751 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-04 10:55:33,751 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,751 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,753 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,753 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,753 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,753 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,754 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,754 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:33,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:33,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:33,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:33,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,755 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,755 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,755 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,755 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-04 10:55:33,755 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,755 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-04 10:55:33,755 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-04 10:55:33,755 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-04 10:55:33,755 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:33,755 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,755 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-04 10:55:33,755 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-04 10:55:33,755 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:33,755 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,755 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-04 10:55:33,755 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-04 10:55:33,755 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-04 10:55:33,755 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-04 10:55:33,755 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:33,756 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:33,756 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:33,756 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:33,756 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. started... 2023-05-04 10:55:33,756 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-04 10:55:33,756 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-04 10:55:33,756 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-04 10:55:33,756 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6ad719aaf381b8be3588a9ac72a2d00d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:33,757 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,758 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,758 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,758 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:33,758 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,758 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:33,764 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf1/24f1f60711474637b28e7dd3f874d630 2023-05-04 10:55:33,774 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf2/66a550f579934d7c9ef87eb2f59ea1df 2023-05-04 10:55:33,778 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf1/24f1f60711474637b28e7dd3f874d630 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/24f1f60711474637b28e7dd3f874d630 2023-05-04 10:55:33,781 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/24f1f60711474637b28e7dd3f874d630, entries=10, sequenceid=45, filesize=5.0 K 2023-05-04 10:55:33,782 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf2/66a550f579934d7c9ef87eb2f59ea1df as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/66a550f579934d7c9ef87eb2f59ea1df 2023-05-04 10:55:33,785 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/66a550f579934d7c9ef87eb2f59ea1df, entries=10, sequenceid=45, filesize=5.0 K 2023-05-04 10:55:33,785 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6ad719aaf381b8be3588a9ac72a2d00d in 29ms, sequenceid=45, compaction requested=true 2023-05-04 10:55:33,785 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6ad719aaf381b8be3588a9ac72a2d00d: 2023-05-04 10:55:33,785 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:33,785 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:33,786 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:33,786 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,786 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-04 10:55:33,786 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-04 10:55:33,787 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,787 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:33,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:33,787 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,787 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:33,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:33,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:33,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:33,789 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-04 10:55:33,789 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3911c56[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:33,789 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-04 10:55:33,789 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,791 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,791 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,791 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,792 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-04 10:55:33,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,792 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,792 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-04 10:55:33,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,792 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-04 10:55:33,792 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-04 10:55:33,792 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:33,792 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-04 10:55:33,792 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-04 10:55:33,792 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:33,792 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:33,792 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:33,793 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,793 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-04 10:55:33,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:33,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:33,793 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,793 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-04 10:55:33,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:33,794 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-04 10:55:33,794 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,794 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-04 10:55:33,794 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,794 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-04 10:55:33,794 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-04 10:55:33,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:33,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:33,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:55:33,797 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:55:33,797 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-04 10:55:33,797 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:33,797 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:33,797 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,797 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:33,798 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,798 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,798 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,798 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:33,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:33,798 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,799 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,798 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,798 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:33,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,799 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,800 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:33,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,801 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,802 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:33,802 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,810 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,810 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,810 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,811 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,811 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,811 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-04 10:55:33,811 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:33,811 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,811 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,811 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,810 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,811 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,811 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,811 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:33,811 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:33,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,811 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,811 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,812 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,812 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,812 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,812 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,812 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,911 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:33,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:33,922 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:33,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:33,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,924 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-04 10:55:33,924 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:33,924 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-04 10:55:33,924 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-04 10:55:33,924 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,924 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,925 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,925 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,925 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,925 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,926 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,926 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:33,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,926 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,926 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,926 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:33,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:33,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,927 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,927 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:33,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,927 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,927 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-04 10:55:33,927 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,927 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-04 10:55:33,927 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:33,927 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-04 10:55:33,927 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-04 10:55:33,927 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-04 10:55:33,928 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-04 10:55:33,928 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,928 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-04 10:55:33,928 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-04 10:55:33,928 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:33,928 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,928 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-04 10:55:33,928 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-04 10:55:33,928 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:33,928 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:33,928 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:33,928 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:33,928 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-04 10:55:33,928 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-04 10:55:33,928 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-04 10:55:33,928 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,928 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. started... 2023-05-04 10:55:33,929 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6ad719aaf381b8be3588a9ac72a2d00d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:33,929 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,929 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:33,931 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,931 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,931 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:33,937 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf1/0ff929e1a4d7400aa565e9eafc77549f 2023-05-04 10:55:33,948 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf2/f93c81ab43ac4ceaa762f92e07d63233 2023-05-04 10:55:33,951 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf1/0ff929e1a4d7400aa565e9eafc77549f as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/0ff929e1a4d7400aa565e9eafc77549f 2023-05-04 10:55:33,954 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/0ff929e1a4d7400aa565e9eafc77549f, entries=10, sequenceid=58, filesize=5.0 K 2023-05-04 10:55:33,955 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf2/f93c81ab43ac4ceaa762f92e07d63233 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/f93c81ab43ac4ceaa762f92e07d63233 2023-05-04 10:55:33,958 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/f93c81ab43ac4ceaa762f92e07d63233, entries=10, sequenceid=58, filesize=5.0 K 2023-05-04 10:55:33,959 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6ad719aaf381b8be3588a9ac72a2d00d in 30ms, sequenceid=58, compaction requested=true 2023-05-04 10:55:33,959 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6ad719aaf381b8be3588a9ac72a2d00d: 2023-05-04 10:55:33,959 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:33,959 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:33,959 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:33,959 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,959 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-04 10:55:33,959 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-04 10:55:33,961 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,961 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:33,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:33,961 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,961 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:33,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:33,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:33,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:33,963 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-04 10:55:33,963 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@421b423f[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:33,963 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-04 10:55:33,963 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,964 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,964 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,964 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,964 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,964 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:33,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,964 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-04 10:55:33,964 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-04 10:55:33,964 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-04 10:55:33,964 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:33,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,964 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-04 10:55:33,964 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-04 10:55:33,964 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:33,964 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-04 10:55:33,964 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:33,965 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-04 10:55:33,965 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,965 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:33,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:33,966 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-04 10:55:33,967 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-04 10:55:33,967 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-04 10:55:33,967 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,967 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,967 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-04 10:55:33,967 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-04 10:55:33,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:33,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:33,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:33,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:55:33,970 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:55:33,970 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-04 10:55:33,970 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:33,970 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:33,970 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,970 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:33,971 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,971 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:33,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:33,971 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,971 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,972 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,972 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,972 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:33,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,973 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,973 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:33,974 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:33,975 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,975 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,976 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,978 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,978 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,978 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,978 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,978 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,978 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:33,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,978 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,978 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-04 10:55:33,979 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:33,978 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,978 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:33,978 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:33,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:33,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,979 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,979 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:33,979 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:33,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,979 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,979 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:33,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,979 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:33,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:33,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:33,979 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:33,979 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:33,979 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:33,980 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,079 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:34,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:34,091 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:34,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:34,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,093 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-04 10:55:34,093 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:34,093 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-04 10:55:34,093 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-04 10:55:34,093 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,093 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,094 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:34,094 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:34,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:34,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:34,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:34,094 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:34,094 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:34,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:34,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:34,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,095 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,095 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:34,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,095 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,095 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,095 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:34,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,096 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:34,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:34,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,096 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-04 10:55:34,096 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,096 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,096 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,096 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-04 10:55:34,096 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:34,096 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-04 10:55:34,096 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-04 10:55:34,096 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,096 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-04 10:55:34,096 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-04 10:55:34,096 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:34,096 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:34,096 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,096 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:34,096 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-04 10:55:34,096 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. started... 2023-05-04 10:55:34,096 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:34,096 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:34,096 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-04 10:55:34,097 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-04 10:55:34,097 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6ad719aaf381b8be3588a9ac72a2d00d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:34,097 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-04 10:55:34,097 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-04 10:55:34,097 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-04 10:55:34,097 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-04 10:55:34,099 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,099 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,099 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,099 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:34,099 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,099 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:34,105 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf1/85cdb00e644a4d21b27fc87ca62d33ad 2023-05-04 10:55:34,115 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf2/02b54b8e9f5a4ddfb08cf58985d108a7 2023-05-04 10:55:34,119 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf1/85cdb00e644a4d21b27fc87ca62d33ad as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/85cdb00e644a4d21b27fc87ca62d33ad 2023-05-04 10:55:34,122 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/85cdb00e644a4d21b27fc87ca62d33ad, entries=10, sequenceid=71, filesize=5.0 K 2023-05-04 10:55:34,122 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf2/02b54b8e9f5a4ddfb08cf58985d108a7 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/02b54b8e9f5a4ddfb08cf58985d108a7 2023-05-04 10:55:34,125 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/02b54b8e9f5a4ddfb08cf58985d108a7, entries=10, sequenceid=71, filesize=5.0 K 2023-05-04 10:55:34,126 INFO [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6ad719aaf381b8be3588a9ac72a2d00d in 29ms, sequenceid=71, compaction requested=true 2023-05-04 10:55:34,126 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6ad719aaf381b8be3588a9ac72a2d00d: 2023-05-04 10:55:34,126 DEBUG [rs(jenkins-hbase4.apache.org,42285,1683197625528)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:34,126 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:34,126 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:34,126 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,126 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-04 10:55:34,126 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-04 10:55:34,128 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,128 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:34,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:34,128 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,128 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:34,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:34,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:34,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:34,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:34,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:34,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-04 10:55:34,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@151d36f5[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:34,130 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-04 10:55:34,130 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,134 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,134 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,135 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-04 10:55:34,135 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-04 10:55:34,135 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:34,134 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,135 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,135 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:34,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,135 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-04 10:55:34,135 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-04 10:55:34,135 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-04 10:55:34,135 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:34,135 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-04 10:55:34,135 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:34,136 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-04 10:55:34,136 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:34,136 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-04 10:55:34,137 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-04 10:55:34,137 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,137 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-04 10:55:34,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:34,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:34,137 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:34,137 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:34,137 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-04 10:55:34,137 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-04 10:55:34,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:34,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:34,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:34,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:34,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:34,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:34,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:34,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,42285,1683197625528': 2023-05-04 10:55:34,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-04 10:55:34,140 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-04 10:55:34,140 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:34,140 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:34,140 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,140 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:34,142 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,142 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,142 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,142 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:34,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:34,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:34,142 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:34,142 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,142 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:34,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:34,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:34,142 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:34,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:34,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:34,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:34,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:34,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:34,143 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:34,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,143 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:34,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:34,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:34,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,145 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:34,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:34,145 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:34,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,145 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:34,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:34,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,147 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,147 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,147 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:34,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:34,147 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:34,147 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-04 10:55:34,148 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:34,147 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:34,148 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:34,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:34,148 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:34,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:34,147 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:34,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:34,148 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:34,148 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:34,147 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:34,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:34,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:34,148 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:34,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:34,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:34,148 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:34,148 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:34,148 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:34,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:34,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:34,149 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,149 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:34,149 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:34,149 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,149 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-04 10:55:34,248 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:34,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:34,256 DEBUG [Listener at localhost/35483] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,257 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=6ad719aaf381b8be3588a9ac72a2d00d, daughterA=a2e336fd219445afffd5859f99c275aa, daughterB=ba6b6253dce8b3861e47b7267a5d66c3 2023-05-04 10:55:34,258 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=6ad719aaf381b8be3588a9ac72a2d00d, daughterA=a2e336fd219445afffd5859f99c275aa, daughterB=ba6b6253dce8b3861e47b7267a5d66c3 2023-05-04 10:55:34,258 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=6ad719aaf381b8be3588a9ac72a2d00d, daughterA=a2e336fd219445afffd5859f99c275aa, daughterB=ba6b6253dce8b3861e47b7267a5d66c3 2023-05-04 10:55:34,258 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=6ad719aaf381b8be3588a9ac72a2d00d, daughterA=a2e336fd219445afffd5859f99c275aa, daughterB=ba6b6253dce8b3861e47b7267a5d66c3 2023-05-04 10:55:34,260 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6ad719aaf381b8be3588a9ac72a2d00d, UNASSIGN}] 2023-05-04 10:55:34,260 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6ad719aaf381b8be3588a9ac72a2d00d, UNASSIGN 2023-05-04 10:55:34,261 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=6ad719aaf381b8be3588a9ac72a2d00d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,261 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683197734261"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197734261"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197734261"}]},"ts":"1683197734261"} 2023-05-04 10:55:34,262 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 6ad719aaf381b8be3588a9ac72a2d00d, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:55:34,413 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6ad719aaf381b8be3588a9ac72a2d00d, disabling compactions & flushes 2023-05-04 10:55:34,414 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:34,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:34,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. after waiting 0 ms 2023-05-04 10:55:34,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:34,414 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 6ad719aaf381b8be3588a9ac72a2d00d 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-04 10:55:34,422 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf1/35012852a1de475e8e6b009247ffacc0 2023-05-04 10:55:34,432 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf2/022825f25fb6438caa296e78ac37067b 2023-05-04 10:55:34,435 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf1/35012852a1de475e8e6b009247ffacc0 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/35012852a1de475e8e6b009247ffacc0 2023-05-04 10:55:34,438 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/35012852a1de475e8e6b009247ffacc0, entries=8, sequenceid=82, filesize=5.0 K 2023-05-04 10:55:34,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.tmp/cf2/022825f25fb6438caa296e78ac37067b as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/022825f25fb6438caa296e78ac37067b 2023-05-04 10:55:34,442 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/022825f25fb6438caa296e78ac37067b, entries=8, sequenceid=82, filesize=5.0 K 2023-05-04 10:55:34,443 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 6ad719aaf381b8be3588a9ac72a2d00d in 29ms, sequenceid=82, compaction requested=true 2023-05-04 10:55:34,447 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-04 10:55:34,447 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:55:34,448 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. 2023-05-04 10:55:34,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6ad719aaf381b8be3588a9ac72a2d00d: 2023-05-04 10:55:34,449 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,450 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=6ad719aaf381b8be3588a9ac72a2d00d, regionState=CLOSED 2023-05-04 10:55:34,450 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683197734450"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197734450"}]},"ts":"1683197734450"} 2023-05-04 10:55:34,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-05-04 10:55:34,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 6ad719aaf381b8be3588a9ac72a2d00d, server=jenkins-hbase4.apache.org,42285,1683197625528 in 189 msec 2023-05-04 10:55:34,454 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-05-04 10:55:34,454 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6ad719aaf381b8be3588a9ac72a2d00d, UNASSIGN in 193 msec 2023-05-04 10:55:34,464 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=6ad719aaf381b8be3588a9ac72a2d00d, threads=14 2023-05-04 10:55:34,465 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/022825f25fb6438caa296e78ac37067b for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,465 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/02b54b8e9f5a4ddfb08cf58985d108a7 for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,465 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/5c04bdb3b4d946ba9e732986652c7b10 for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,465 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/66a550f579934d7c9ef87eb2f59ea1df for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,465 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/8ad09df92c1d4d9f8445ce3e1c47713d for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,466 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/e794281b4afe4306b7cdf8935162548f for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,466 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/f93c81ab43ac4ceaa762f92e07d63233 for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,472 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/0ff929e1a4d7400aa565e9eafc77549f for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,472 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/24f1f60711474637b28e7dd3f874d630 for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,472 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/35012852a1de475e8e6b009247ffacc0 for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,473 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/85ba15d7901e41b0813ccf7e359d41ff for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,473 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/85cdb00e644a4d21b27fc87ca62d33ad for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,473 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/cace4ee073a54636a9ade830c70d1e3e for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,473 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/d82bab81d0784a8ea602e56ce8a8e720 for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,475 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/8ad09df92c1d4d9f8445ce3e1c47713d, top=false 2023-05-04 10:55:34,477 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/66a550f579934d7c9ef87eb2f59ea1df, top=false 2023-05-04 10:55:34,481 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/24f1f60711474637b28e7dd3f874d630, top=false 2023-05-04 10:55:34,486 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/022825f25fb6438caa296e78ac37067b, top=false 2023-05-04 10:55:34,487 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/85ba15d7901e41b0813ccf7e359d41ff, top=false 2023-05-04 10:55:34,491 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/02b54b8e9f5a4ddfb08cf58985d108a7, top=false 2023-05-04 10:55:34,491 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/f93c81ab43ac4ceaa762f92e07d63233, top=false 2023-05-04 10:55:34,491 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/5c04bdb3b4d946ba9e732986652c7b10, top=false 2023-05-04 10:55:34,492 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/e794281b4afe4306b7cdf8935162548f, top=false 2023-05-04 10:55:34,492 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/cace4ee073a54636a9ade830c70d1e3e, top=false 2023-05-04 10:55:34,492 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/35012852a1de475e8e6b009247ffacc0, top=false 2023-05-04 10:55:34,493 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/85cdb00e644a4d21b27fc87ca62d33ad, top=false 2023-05-04 10:55:34,493 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/0ff929e1a4d7400aa565e9eafc77549f, top=false 2023-05-04 10:55:34,497 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/d82bab81d0784a8ea602e56ce8a8e720, top=false 2023-05-04 10:55:34,504 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.splits/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-66a550f579934d7c9ef87eb2f59ea1df for child: a2e336fd219445afffd5859f99c275aa, parent: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,507 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/66a550f579934d7c9ef87eb2f59ea1df for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,520 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.splits/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-022825f25fb6438caa296e78ac37067b for child: a2e336fd219445afffd5859f99c275aa, parent: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,521 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.splits/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-85ba15d7901e41b0813ccf7e359d41ff for child: a2e336fd219445afffd5859f99c275aa, parent: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,521 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.splits/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-24f1f60711474637b28e7dd3f874d630 for child: a2e336fd219445afffd5859f99c275aa, parent: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,521 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.splits/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-5c04bdb3b4d946ba9e732986652c7b10 for child: a2e336fd219445afffd5859f99c275aa, parent: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,524 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.splits/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-0ff929e1a4d7400aa565e9eafc77549f for child: a2e336fd219445afffd5859f99c275aa, parent: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,527 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/022825f25fb6438caa296e78ac37067b for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,527 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/85ba15d7901e41b0813ccf7e359d41ff for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,529 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/24f1f60711474637b28e7dd3f874d630 for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,529 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/0ff929e1a4d7400aa565e9eafc77549f for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,529 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.splits/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-8ad09df92c1d4d9f8445ce3e1c47713d for child: a2e336fd219445afffd5859f99c275aa, parent: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,529 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/5c04bdb3b4d946ba9e732986652c7b10 for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,529 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.splits/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-85cdb00e644a4d21b27fc87ca62d33ad for child: a2e336fd219445afffd5859f99c275aa, parent: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,529 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.splits/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-f93c81ab43ac4ceaa762f92e07d63233 for child: a2e336fd219445afffd5859f99c275aa, parent: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,529 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.splits/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-cace4ee073a54636a9ade830c70d1e3e for child: a2e336fd219445afffd5859f99c275aa, parent: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,532 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/8ad09df92c1d4d9f8445ce3e1c47713d for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,532 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/85cdb00e644a4d21b27fc87ca62d33ad for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,533 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/f93c81ab43ac4ceaa762f92e07d63233 for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,533 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/cace4ee073a54636a9ade830c70d1e3e for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,536 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.splits/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-35012852a1de475e8e6b009247ffacc0 for child: a2e336fd219445afffd5859f99c275aa, parent: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,536 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.splits/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-02b54b8e9f5a4ddfb08cf58985d108a7 for child: a2e336fd219445afffd5859f99c275aa, parent: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,536 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.splits/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-d82bab81d0784a8ea602e56ce8a8e720 for child: a2e336fd219445afffd5859f99c275aa, parent: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,537 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/.splits/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-e794281b4afe4306b7cdf8935162548f for child: a2e336fd219445afffd5859f99c275aa, parent: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,539 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/35012852a1de475e8e6b009247ffacc0 for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,539 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/d82bab81d0784a8ea602e56ce8a8e720 for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,540 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/e794281b4afe4306b7cdf8935162548f for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,540 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/02b54b8e9f5a4ddfb08cf58985d108a7 for region: 6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:34,540 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 6ad719aaf381b8be3588a9ac72a2d00d Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-05-04 10:55:34,579 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-04 10:55:34,581 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/ba6b6253dce8b3861e47b7267a5d66c3/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-04 10:55:34,583 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683197734582"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1683197734582"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1683197734582"}]},"ts":"1683197734582"} 2023-05-04 10:55:34,583 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683197734582"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197734582"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197734582"}]},"ts":"1683197734582"} 2023-05-04 10:55:34,583 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1683197734256.ba6b6253dce8b3861e47b7267a5d66c3.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683197734582"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197734582"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197734582"}]},"ts":"1683197734582"} 2023-05-04 10:55:34,586 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a2e336fd219445afffd5859f99c275aa, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=ba6b6253dce8b3861e47b7267a5d66c3, ASSIGN}] 2023-05-04 10:55:34,586 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=ba6b6253dce8b3861e47b7267a5d66c3, ASSIGN 2023-05-04 10:55:34,587 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=ba6b6253dce8b3861e47b7267a5d66c3, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42285,1683197625528; forceNewPlan=false, retain=false 2023-05-04 10:55:34,588 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a2e336fd219445afffd5859f99c275aa, ASSIGN 2023-05-04 10:55:34,588 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a2e336fd219445afffd5859f99c275aa, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,42285,1683197625528; forceNewPlan=false, retain=false 2023-05-04 10:55:34,737 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-04 10:55:34,739 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=a2e336fd219445afffd5859f99c275aa, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,739 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683197734739"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197734739"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197734739"}]},"ts":"1683197734739"} 2023-05-04 10:55:34,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=131, state=RUNNABLE; OpenRegionProcedure a2e336fd219445afffd5859f99c275aa, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:55:34,743 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=ba6b6253dce8b3861e47b7267a5d66c3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,743 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1683197734256.ba6b6253dce8b3861e47b7267a5d66c3.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683197734743"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197734743"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197734743"}]},"ts":"1683197734743"} 2023-05-04 10:55:34,744 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=132, state=RUNNABLE; OpenRegionProcedure ba6b6253dce8b3861e47b7267a5d66c3, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:55:34,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,71,1683197734256.ba6b6253dce8b3861e47b7267a5d66c3. 2023-05-04 10:55:34,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ba6b6253dce8b3861e47b7267a5d66c3, NAME => 'testSplitTableRegionEmptyDaughter,71,1683197734256.ba6b6253dce8b3861e47b7267a5d66c3.', STARTKEY => '71', ENDKEY => ''} 2023-05-04 10:55:34,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:55:34,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter ba6b6253dce8b3861e47b7267a5d66c3 2023-05-04 10:55:34,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,71,1683197734256.ba6b6253dce8b3861e47b7267a5d66c3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:55:34,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ba6b6253dce8b3861e47b7267a5d66c3 2023-05-04 10:55:34,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ba6b6253dce8b3861e47b7267a5d66c3 2023-05-04 10:55:34,897 INFO [StoreOpener-ba6b6253dce8b3861e47b7267a5d66c3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ba6b6253dce8b3861e47b7267a5d66c3 2023-05-04 10:55:34,898 DEBUG [StoreOpener-ba6b6253dce8b3861e47b7267a5d66c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/ba6b6253dce8b3861e47b7267a5d66c3/cf1 2023-05-04 10:55:34,899 DEBUG [StoreOpener-ba6b6253dce8b3861e47b7267a5d66c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/ba6b6253dce8b3861e47b7267a5d66c3/cf1 2023-05-04 10:55:34,899 INFO [StoreOpener-ba6b6253dce8b3861e47b7267a5d66c3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ba6b6253dce8b3861e47b7267a5d66c3 columnFamilyName cf1 2023-05-04 10:55:34,899 INFO [StoreOpener-ba6b6253dce8b3861e47b7267a5d66c3-1] regionserver.HStore(310): Store=ba6b6253dce8b3861e47b7267a5d66c3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:34,900 INFO [StoreOpener-ba6b6253dce8b3861e47b7267a5d66c3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ba6b6253dce8b3861e47b7267a5d66c3 2023-05-04 10:55:34,901 DEBUG [StoreOpener-ba6b6253dce8b3861e47b7267a5d66c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/ba6b6253dce8b3861e47b7267a5d66c3/cf2 2023-05-04 10:55:34,901 DEBUG [StoreOpener-ba6b6253dce8b3861e47b7267a5d66c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/ba6b6253dce8b3861e47b7267a5d66c3/cf2 2023-05-04 10:55:34,901 INFO [StoreOpener-ba6b6253dce8b3861e47b7267a5d66c3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ba6b6253dce8b3861e47b7267a5d66c3 columnFamilyName cf2 2023-05-04 10:55:34,902 INFO [StoreOpener-ba6b6253dce8b3861e47b7267a5d66c3-1] regionserver.HStore(310): Store=ba6b6253dce8b3861e47b7267a5d66c3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:34,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/ba6b6253dce8b3861e47b7267a5d66c3 2023-05-04 10:55:34,903 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/ba6b6253dce8b3861e47b7267a5d66c3 2023-05-04 10:55:34,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:55:34,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ba6b6253dce8b3861e47b7267a5d66c3 2023-05-04 10:55:34,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ba6b6253dce8b3861e47b7267a5d66c3; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65891500, jitterRate=-0.018140137195587158}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:55:34,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ba6b6253dce8b3861e47b7267a5d66c3: 2023-05-04 10:55:34,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1683197734256.ba6b6253dce8b3861e47b7267a5d66c3., pid=134, masterSystemTime=1683197734892 2023-05-04 10:55:34,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1683197734256.ba6b6253dce8b3861e47b7267a5d66c3. 2023-05-04 10:55:34,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,71,1683197734256.ba6b6253dce8b3861e47b7267a5d66c3. 2023-05-04 10:55:34,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa. 2023-05-04 10:55:34,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a2e336fd219445afffd5859f99c275aa, NAME => 'testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.', STARTKEY => '', ENDKEY => '71'} 2023-05-04 10:55:34,909 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=ba6b6253dce8b3861e47b7267a5d66c3, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:55:34,909 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1683197734256.ba6b6253dce8b3861e47b7267a5d66c3.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683197734909"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197734909"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197734909"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197734909"}]},"ts":"1683197734909"} 2023-05-04 10:55:34,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter a2e336fd219445afffd5859f99c275aa 2023-05-04 10:55:34,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:55:34,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a2e336fd219445afffd5859f99c275aa 2023-05-04 10:55:34,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a2e336fd219445afffd5859f99c275aa 2023-05-04 10:55:34,910 INFO [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a2e336fd219445afffd5859f99c275aa 2023-05-04 10:55:34,911 DEBUG [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1 2023-05-04 10:55:34,911 DEBUG [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1 2023-05-04 10:55:34,912 INFO [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a2e336fd219445afffd5859f99c275aa columnFamilyName cf1 2023-05-04 10:55:34,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=132 2023-05-04 10:55:34,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=132, state=SUCCESS; OpenRegionProcedure ba6b6253dce8b3861e47b7267a5d66c3, server=jenkins-hbase4.apache.org,42285,1683197625528 in 166 msec 2023-05-04 10:55:34,913 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=ba6b6253dce8b3861e47b7267a5d66c3, ASSIGN in 326 msec 2023-05-04 10:55:34,917 DEBUG [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-0ff929e1a4d7400aa565e9eafc77549f 2023-05-04 10:55:34,920 DEBUG [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-24f1f60711474637b28e7dd3f874d630 2023-05-04 10:55:34,923 DEBUG [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-35012852a1de475e8e6b009247ffacc0 2023-05-04 10:55:34,926 DEBUG [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-85ba15d7901e41b0813ccf7e359d41ff 2023-05-04 10:55:34,928 DEBUG [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-85cdb00e644a4d21b27fc87ca62d33ad 2023-05-04 10:55:34,931 DEBUG [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-cace4ee073a54636a9ade830c70d1e3e 2023-05-04 10:55:34,933 DEBUG [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-d82bab81d0784a8ea602e56ce8a8e720 2023-05-04 10:55:34,933 INFO [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] regionserver.HStore(310): Store=a2e336fd219445afffd5859f99c275aa/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:34,933 INFO [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a2e336fd219445afffd5859f99c275aa 2023-05-04 10:55:34,934 DEBUG [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2 2023-05-04 10:55:34,934 DEBUG [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2 2023-05-04 10:55:34,934 INFO [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a2e336fd219445afffd5859f99c275aa columnFamilyName cf2 2023-05-04 10:55:34,938 DEBUG [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-022825f25fb6438caa296e78ac37067b 2023-05-04 10:55:34,941 DEBUG [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-02b54b8e9f5a4ddfb08cf58985d108a7 2023-05-04 10:55:34,943 DEBUG [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-5c04bdb3b4d946ba9e732986652c7b10 2023-05-04 10:55:34,945 DEBUG [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-66a550f579934d7c9ef87eb2f59ea1df 2023-05-04 10:55:34,948 DEBUG [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-8ad09df92c1d4d9f8445ce3e1c47713d 2023-05-04 10:55:34,950 DEBUG [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-e794281b4afe4306b7cdf8935162548f 2023-05-04 10:55:34,952 DEBUG [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-f93c81ab43ac4ceaa762f92e07d63233 2023-05-04 10:55:34,953 INFO [StoreOpener-a2e336fd219445afffd5859f99c275aa-1] regionserver.HStore(310): Store=a2e336fd219445afffd5859f99c275aa/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:34,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa 2023-05-04 10:55:34,954 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa 2023-05-04 10:55:34,955 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:55:34,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a2e336fd219445afffd5859f99c275aa 2023-05-04 10:55:34,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a2e336fd219445afffd5859f99c275aa; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63489937, jitterRate=-0.05392621457576752}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:55:34,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a2e336fd219445afffd5859f99c275aa: 2023-05-04 10:55:34,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa., pid=133, masterSystemTime=1683197734892 2023-05-04 10:55:34,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:55:34,959 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:55:34,959 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:55:34,959 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-04 10:55:34,959 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-05-04 10:55:34,959 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-05-04 10:55:34,959 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1912): a2e336fd219445afffd5859f99c275aa/cf1 is initiating minor compaction (all files) 2023-05-04 10:55:34,959 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1912): a2e336fd219445afffd5859f99c275aa/cf2 is initiating minor compaction (all files) 2023-05-04 10:55:34,959 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a2e336fd219445afffd5859f99c275aa/cf1 in testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa. 2023-05-04 10:55:34,959 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HRegion(2259): Starting compaction of a2e336fd219445afffd5859f99c275aa/cf2 in testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa. 2023-05-04 10:55:34,959 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-d82bab81d0784a8ea602e56ce8a8e720, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-85ba15d7901e41b0813ccf7e359d41ff, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-cace4ee073a54636a9ade830c70d1e3e, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-24f1f60711474637b28e7dd3f874d630, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-0ff929e1a4d7400aa565e9eafc77549f, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-85cdb00e644a4d21b27fc87ca62d33ad, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-35012852a1de475e8e6b009247ffacc0] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/.tmp, totalSize=34.9 K 2023-05-04 10:55:34,959 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-5c04bdb3b4d946ba9e732986652c7b10, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-8ad09df92c1d4d9f8445ce3e1c47713d, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-e794281b4afe4306b7cdf8935162548f, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-66a550f579934d7c9ef87eb2f59ea1df, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-f93c81ab43ac4ceaa762f92e07d63233, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-02b54b8e9f5a4ddfb08cf58985d108a7, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-022825f25fb6438caa296e78ac37067b] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/.tmp, totalSize=34.9 K 2023-05-04 10:55:34,959 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-d82bab81d0784a8ea602e56ce8a8e720, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683197732383 2023-05-04 10:55:34,960 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-5c04bdb3b4d946ba9e732986652c7b10, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683197732383 2023-05-04 10:55:34,960 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-85ba15d7901e41b0813ccf7e359d41ff, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683197733356 2023-05-04 10:55:34,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa. 2023-05-04 10:55:34,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa. 2023-05-04 10:55:34,961 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-8ad09df92c1d4d9f8445ce3e1c47713d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683197733356 2023-05-04 10:55:34,961 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-cace4ee073a54636a9ade830c70d1e3e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683197733556 2023-05-04 10:55:34,961 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-e794281b4afe4306b7cdf8935162548f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683197733556 2023-05-04 10:55:34,961 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=a2e336fd219445afffd5859f99c275aa, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:34,961 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-24f1f60711474637b28e7dd3f874d630, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683197733740 2023-05-04 10:55:34,961 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683197734961"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197734961"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197734961"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197734961"}]},"ts":"1683197734961"} 2023-05-04 10:55:34,961 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-66a550f579934d7c9ef87eb2f59ea1df, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683197733740 2023-05-04 10:55:34,961 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-0ff929e1a4d7400aa565e9eafc77549f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683197733913 2023-05-04 10:55:34,962 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-f93c81ab43ac4ceaa762f92e07d63233, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683197733913 2023-05-04 10:55:34,962 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-85cdb00e644a4d21b27fc87ca62d33ad, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683197734081 2023-05-04 10:55:34,962 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-02b54b8e9f5a4ddfb08cf58985d108a7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683197734081 2023-05-04 10:55:34,962 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-35012852a1de475e8e6b009247ffacc0, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683197734250 2023-05-04 10:55:34,962 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-022825f25fb6438caa296e78ac37067b, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683197734250 2023-05-04 10:55:34,964 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=131 2023-05-04 10:55:34,964 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=131, state=SUCCESS; OpenRegionProcedure a2e336fd219445afffd5859f99c275aa, server=jenkins-hbase4.apache.org,42285,1683197625528 in 221 msec 2023-05-04 10:55:34,966 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-05-04 10:55:34,966 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a2e336fd219445afffd5859f99c275aa, ASSIGN in 378 msec 2023-05-04 10:55:34,967 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=6ad719aaf381b8be3588a9ac72a2d00d, daughterA=a2e336fd219445afffd5859f99c275aa, daughterB=ba6b6253dce8b3861e47b7267a5d66c3 in 710 msec 2023-05-04 10:55:34,988 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] throttle.PressureAwareThroughputController(145): a2e336fd219445afffd5859f99c275aa#cf2#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:55:34,988 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] throttle.PressureAwareThroughputController(145): a2e336fd219445afffd5859f99c275aa#cf1#compaction#127 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:55:35,003 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/.tmp/cf2/c8f38be598fe48499bb6cccf58310fab as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/c8f38be598fe48499bb6cccf58310fab 2023-05-04 10:55:35,007 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a2e336fd219445afffd5859f99c275aa/cf2 of a2e336fd219445afffd5859f99c275aa into c8f38be598fe48499bb6cccf58310fab(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:55:35,008 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for a2e336fd219445afffd5859f99c275aa: 2023-05-04 10:55:35,008 INFO [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa., storeName=a2e336fd219445afffd5859f99c275aa/cf2, priority=9, startTime=1683197734959; duration=0sec 2023-05-04 10:55:35,009 DEBUG [RS:2;jenkins-hbase4:42285-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:55:35,009 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42285] ipc.CallRunner(144): callId: 1017 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:37016 deadline: 1683197795009, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d. is not online on jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:35,093 ERROR [BP-2139253951-172.31.14.131-1683197620399 heartbeating to localhost/127.0.0.1:44245] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2139253951-172.31.14.131-1683197620399 (Datanode Uuid 9432e34f-f582-4227-a602-1b8f52f4fa10) service to localhost/127.0.0.1:44245 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-05-04 10:55:35,126 DEBUG [Listener at localhost/35483] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-05-04 10:55:35,130 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:55:35,131 INFO [Listener at localhost/35483] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-05-04 10:55:35,131 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-05-04 10:55:35,132 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-04 10:55:35,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-05-04 10:55:35,135 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197735135"}]},"ts":"1683197735135"} 2023-05-04 10:55:35,136 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-05-04 10:55:35,137 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-05-04 10:55:35,138 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a2e336fd219445afffd5859f99c275aa, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=ba6b6253dce8b3861e47b7267a5d66c3, UNASSIGN}] 2023-05-04 10:55:35,138 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=ba6b6253dce8b3861e47b7267a5d66c3, UNASSIGN 2023-05-04 10:55:35,139 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=ba6b6253dce8b3861e47b7267a5d66c3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:35,139 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1683197734256.ba6b6253dce8b3861e47b7267a5d66c3.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683197735139"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197735139"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197735139"}]},"ts":"1683197735139"} 2023-05-04 10:55:35,140 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure ba6b6253dce8b3861e47b7267a5d66c3, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:55:35,142 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a2e336fd219445afffd5859f99c275aa, UNASSIGN 2023-05-04 10:55:35,143 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=a2e336fd219445afffd5859f99c275aa, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:35,143 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683197735143"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197735143"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197735143"}]},"ts":"1683197735143"} 2023-05-04 10:55:35,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure a2e336fd219445afffd5859f99c275aa, server=jenkins-hbase4.apache.org,42285,1683197625528}] 2023-05-04 10:55:35,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-05-04 10:55:35,292 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close a2e336fd219445afffd5859f99c275aa 2023-05-04 10:55:35,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a2e336fd219445afffd5859f99c275aa, disabling compactions & flushes 2023-05-04 10:55:35,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa. 2023-05-04 10:55:35,404 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/.tmp/cf1/bb6ceb0a379c4fadb498344038360624 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/bb6ceb0a379c4fadb498344038360624 2023-05-04 10:55:35,408 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a2e336fd219445afffd5859f99c275aa/cf1 of a2e336fd219445afffd5859f99c275aa into bb6ceb0a379c4fadb498344038360624(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:55:35,408 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a2e336fd219445afffd5859f99c275aa: 2023-05-04 10:55:35,408 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa. 2023-05-04 10:55:35,408 INFO [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa., storeName=a2e336fd219445afffd5859f99c275aa/cf1, priority=9, startTime=1683197734958; duration=0sec 2023-05-04 10:55:35,408 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa. 2023-05-04 10:55:35,408 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa. after waiting 0 ms 2023-05-04 10:55:35,408 DEBUG [RS:2;jenkins-hbase4:42285-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:55:35,408 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa. 2023-05-04 10:55:35,409 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-d82bab81d0784a8ea602e56ce8a8e720, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-85ba15d7901e41b0813ccf7e359d41ff, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-cace4ee073a54636a9ade830c70d1e3e, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-24f1f60711474637b28e7dd3f874d630, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-0ff929e1a4d7400aa565e9eafc77549f, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-85cdb00e644a4d21b27fc87ca62d33ad, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-35012852a1de475e8e6b009247ffacc0] to archive 2023-05-04 10:55:35,409 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:55:35,411 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-d82bab81d0784a8ea602e56ce8a8e720 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-d82bab81d0784a8ea602e56ce8a8e720 2023-05-04 10:55:35,412 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-85ba15d7901e41b0813ccf7e359d41ff to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-85ba15d7901e41b0813ccf7e359d41ff 2023-05-04 10:55:35,414 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-cace4ee073a54636a9ade830c70d1e3e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-cace4ee073a54636a9ade830c70d1e3e 2023-05-04 10:55:35,415 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-24f1f60711474637b28e7dd3f874d630 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-24f1f60711474637b28e7dd3f874d630 2023-05-04 10:55:35,416 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-0ff929e1a4d7400aa565e9eafc77549f to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-0ff929e1a4d7400aa565e9eafc77549f 2023-05-04 10:55:35,417 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-85cdb00e644a4d21b27fc87ca62d33ad to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-85cdb00e644a4d21b27fc87ca62d33ad 2023-05-04 10:55:35,418 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-35012852a1de475e8e6b009247ffacc0 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-35012852a1de475e8e6b009247ffacc0 2023-05-04 10:55:35,419 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-5c04bdb3b4d946ba9e732986652c7b10, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-8ad09df92c1d4d9f8445ce3e1c47713d, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-e794281b4afe4306b7cdf8935162548f, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-66a550f579934d7c9ef87eb2f59ea1df, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-f93c81ab43ac4ceaa762f92e07d63233, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-02b54b8e9f5a4ddfb08cf58985d108a7, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-022825f25fb6438caa296e78ac37067b] to archive 2023-05-04 10:55:35,420 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:55:35,421 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-5c04bdb3b4d946ba9e732986652c7b10 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-5c04bdb3b4d946ba9e732986652c7b10 2023-05-04 10:55:35,422 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-8ad09df92c1d4d9f8445ce3e1c47713d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-8ad09df92c1d4d9f8445ce3e1c47713d 2023-05-04 10:55:35,423 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-e794281b4afe4306b7cdf8935162548f to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-e794281b4afe4306b7cdf8935162548f 2023-05-04 10:55:35,424 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-66a550f579934d7c9ef87eb2f59ea1df to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-66a550f579934d7c9ef87eb2f59ea1df 2023-05-04 10:55:35,424 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-f93c81ab43ac4ceaa762f92e07d63233 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-f93c81ab43ac4ceaa762f92e07d63233 2023-05-04 10:55:35,425 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-02b54b8e9f5a4ddfb08cf58985d108a7 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-02b54b8e9f5a4ddfb08cf58985d108a7 2023-05-04 10:55:35,426 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-022825f25fb6438caa296e78ac37067b to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/testSplitTableRegionEmptyDaughter=6ad719aaf381b8be3588a9ac72a2d00d-022825f25fb6438caa296e78ac37067b 2023-05-04 10:55:35,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-04 10:55:35,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:55:35,430 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa. 2023-05-04 10:55:35,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a2e336fd219445afffd5859f99c275aa: 2023-05-04 10:55:35,432 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed a2e336fd219445afffd5859f99c275aa 2023-05-04 10:55:35,432 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close ba6b6253dce8b3861e47b7267a5d66c3 2023-05-04 10:55:35,433 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ba6b6253dce8b3861e47b7267a5d66c3, disabling compactions & flushes 2023-05-04 10:55:35,433 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,71,1683197734256.ba6b6253dce8b3861e47b7267a5d66c3. 2023-05-04 10:55:35,433 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1683197734256.ba6b6253dce8b3861e47b7267a5d66c3. 2023-05-04 10:55:35,433 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1683197734256.ba6b6253dce8b3861e47b7267a5d66c3. after waiting 0 ms 2023-05-04 10:55:35,433 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1683197734256.ba6b6253dce8b3861e47b7267a5d66c3. 2023-05-04 10:55:35,433 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=a2e336fd219445afffd5859f99c275aa, regionState=CLOSED 2023-05-04 10:55:35,433 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683197735433"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197735433"}]},"ts":"1683197735433"} 2023-05-04 10:55:35,436 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegionEmptyDaughter/ba6b6253dce8b3861e47b7267a5d66c3/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-05-04 10:55:35,436 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:55:35,436 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-05-04 10:55:35,436 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,71,1683197734256.ba6b6253dce8b3861e47b7267a5d66c3. 2023-05-04 10:55:35,436 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ba6b6253dce8b3861e47b7267a5d66c3: 2023-05-04 10:55:35,436 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure a2e336fd219445afffd5859f99c275aa, server=jenkins-hbase4.apache.org,42285,1683197625528 in 291 msec 2023-05-04 10:55:35,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a2e336fd219445afffd5859f99c275aa, UNASSIGN in 298 msec 2023-05-04 10:55:35,437 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed ba6b6253dce8b3861e47b7267a5d66c3 2023-05-04 10:55:35,438 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=ba6b6253dce8b3861e47b7267a5d66c3, regionState=CLOSED 2023-05-04 10:55:35,438 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1683197734256.ba6b6253dce8b3861e47b7267a5d66c3.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683197735438"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197735438"}]},"ts":"1683197735438"} 2023-05-04 10:55:35,441 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-05-04 10:55:35,441 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure ba6b6253dce8b3861e47b7267a5d66c3, server=jenkins-hbase4.apache.org,42285,1683197625528 in 299 msec 2023-05-04 10:55:35,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-05-04 10:55:35,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-05-04 10:55:35,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=ba6b6253dce8b3861e47b7267a5d66c3, UNASSIGN in 303 msec 2023-05-04 10:55:35,443 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197735443"}]},"ts":"1683197735443"} 2023-05-04 10:55:35,444 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-05-04 10:55:35,445 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-05-04 10:55:35,447 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 315 msec 2023-05-04 10:55:35,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-05-04 10:55:35,743 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-05-04 10:55:35,744 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-05-04 10:55:35,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-04 10:55:35,746 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-04 10:55:35,747 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-04 10:55:35,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-05-04 10:55:35,751 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:35,751 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa 2023-05-04 10:55:35,751 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/ba6b6253dce8b3861e47b7267a5d66c3 2023-05-04 10:55:35,754 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/recovered.edits] 2023-05-04 10:55:35,754 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/ba6b6253dce8b3861e47b7267a5d66c3/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/ba6b6253dce8b3861e47b7267a5d66c3/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/ba6b6253dce8b3861e47b7267a5d66c3/recovered.edits] 2023-05-04 10:55:35,754 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/recovered.edits] 2023-05-04 10:55:35,757 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/bb6ceb0a379c4fadb498344038360624 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf1/bb6ceb0a379c4fadb498344038360624 2023-05-04 10:55:35,757 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/0ff929e1a4d7400aa565e9eafc77549f to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/0ff929e1a4d7400aa565e9eafc77549f 2023-05-04 10:55:35,759 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/24f1f60711474637b28e7dd3f874d630 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/24f1f60711474637b28e7dd3f874d630 2023-05-04 10:55:35,760 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/c8f38be598fe48499bb6cccf58310fab to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/cf2/c8f38be598fe48499bb6cccf58310fab 2023-05-04 10:55:35,760 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/ba6b6253dce8b3861e47b7267a5d66c3/recovered.edits/88.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/ba6b6253dce8b3861e47b7267a5d66c3/recovered.edits/88.seqid 2023-05-04 10:55:35,761 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/35012852a1de475e8e6b009247ffacc0 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/35012852a1de475e8e6b009247ffacc0 2023-05-04 10:55:35,761 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/ba6b6253dce8b3861e47b7267a5d66c3 2023-05-04 10:55:35,762 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/85ba15d7901e41b0813ccf7e359d41ff to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/85ba15d7901e41b0813ccf7e359d41ff 2023-05-04 10:55:35,763 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/recovered.edits/90.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa/recovered.edits/90.seqid 2023-05-04 10:55:35,763 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/a2e336fd219445afffd5859f99c275aa 2023-05-04 10:55:35,764 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/85cdb00e644a4d21b27fc87ca62d33ad to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/85cdb00e644a4d21b27fc87ca62d33ad 2023-05-04 10:55:35,765 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/cace4ee073a54636a9ade830c70d1e3e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/cace4ee073a54636a9ade830c70d1e3e 2023-05-04 10:55:35,766 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/d82bab81d0784a8ea602e56ce8a8e720 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf1/d82bab81d0784a8ea602e56ce8a8e720 2023-05-04 10:55:35,768 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/022825f25fb6438caa296e78ac37067b to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/022825f25fb6438caa296e78ac37067b 2023-05-04 10:55:35,770 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/02b54b8e9f5a4ddfb08cf58985d108a7 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/02b54b8e9f5a4ddfb08cf58985d108a7 2023-05-04 10:55:35,771 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/5c04bdb3b4d946ba9e732986652c7b10 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/5c04bdb3b4d946ba9e732986652c7b10 2023-05-04 10:55:35,772 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/66a550f579934d7c9ef87eb2f59ea1df to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/66a550f579934d7c9ef87eb2f59ea1df 2023-05-04 10:55:35,773 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/8ad09df92c1d4d9f8445ce3e1c47713d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/8ad09df92c1d4d9f8445ce3e1c47713d 2023-05-04 10:55:35,775 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/e794281b4afe4306b7cdf8935162548f to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/e794281b4afe4306b7cdf8935162548f 2023-05-04 10:55:35,776 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/f93c81ab43ac4ceaa762f92e07d63233 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/cf2/f93c81ab43ac4ceaa762f92e07d63233 2023-05-04 10:55:35,779 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/recovered.edits/85.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d/recovered.edits/85.seqid 2023-05-04 10:55:35,779 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegionEmptyDaughter/6ad719aaf381b8be3588a9ac72a2d00d 2023-05-04 10:55:35,779 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-05-04 10:55:35,781 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-04 10:55:35,783 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-05-04 10:55:35,785 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-05-04 10:55:35,786 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-04 10:55:35,786 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-05-04 10:55:35,786 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197735786"}]},"ts":"9223372036854775807"} 2023-05-04 10:55:35,786 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197735786"}]},"ts":"9223372036854775807"} 2023-05-04 10:55:35,786 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1683197734256.ba6b6253dce8b3861e47b7267a5d66c3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197735786"}]},"ts":"9223372036854775807"} 2023-05-04 10:55:35,789 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-04 10:55:35,789 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 6ad719aaf381b8be3588a9ac72a2d00d, NAME => 'testSplitTableRegionEmptyDaughter,,1683197731880.6ad719aaf381b8be3588a9ac72a2d00d.', STARTKEY => '', ENDKEY => ''}, {ENCODED => a2e336fd219445afffd5859f99c275aa, NAME => 'testSplitTableRegionEmptyDaughter,,1683197734256.a2e336fd219445afffd5859f99c275aa.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => ba6b6253dce8b3861e47b7267a5d66c3, NAME => 'testSplitTableRegionEmptyDaughter,71,1683197734256.ba6b6253dce8b3861e47b7267a5d66c3.', STARTKEY => '71', ENDKEY => ''}] 2023-05-04 10:55:35,789 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-05-04 10:55:35,789 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683197735789"}]},"ts":"9223372036854775807"} 2023-05-04 10:55:35,790 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-05-04 10:55:35,793 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-04 10:55:35,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 49 msec 2023-05-04 10:55:35,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-05-04 10:55:35,848 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-05-04 10:55:35,862 INFO [Listener at localhost/35483] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=374 (was 375), OpenFileDescriptor=829 (was 829), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=177 (was 175) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=6312 (was 6312) 2023-05-04 10:55:35,876 INFO [Listener at localhost/35483] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=374, OpenFileDescriptor=829, MaxFileDescriptor=60000, SystemLoadAverage=177, ProcessCount=170, AvailableMemoryMB=6311 2023-05-04 10:55:35,876 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:55:35,878 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-04 10:55:35,878 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-04 10:55:35,879 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-05-04 10:55:35,881 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-05-04 10:55:35,882 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-04 10:55:35,883 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:35,883 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d empty. 2023-05-04 10:55:35,884 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:35,884 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-05-04 10:55:35,895 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-05-04 10:55:35,897 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => e26c215b73ee55a3dcea7aacd4b72b4d, NAME => 'testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp 2023-05-04 10:55:35,903 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:55:35,903 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1604): Closing e26c215b73ee55a3dcea7aacd4b72b4d, disabling compactions & flushes 2023-05-04 10:55:35,903 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:35,903 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:35,903 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. after waiting 0 ms 2023-05-04 10:55:35,903 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:35,904 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:35,904 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1558): Region close journal for e26c215b73ee55a3dcea7aacd4b72b4d: 2023-05-04 10:55:35,905 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-05-04 10:55:35,906 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683197735906"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197735906"}]},"ts":"1683197735906"} 2023-05-04 10:55:35,907 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-04 10:55:35,908 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-04 10:55:35,908 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197735908"}]},"ts":"1683197735908"} 2023-05-04 10:55:35,909 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-05-04 10:55:35,911 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-04 10:55:35,912 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-04 10:55:35,912 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-04 10:55:35,912 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-04 10:55:35,912 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-04 10:55:35,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=e26c215b73ee55a3dcea7aacd4b72b4d, ASSIGN}] 2023-05-04 10:55:35,913 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=e26c215b73ee55a3dcea7aacd4b72b4d, ASSIGN 2023-05-04 10:55:35,914 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=e26c215b73ee55a3dcea7aacd4b72b4d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33581,1683197625489; forceNewPlan=false, retain=false 2023-05-04 10:55:36,064 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-04 10:55:36,065 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=e26c215b73ee55a3dcea7aacd4b72b4d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,065 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683197736065"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197736065"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197736065"}]},"ts":"1683197736065"} 2023-05-04 10:55:36,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure e26c215b73ee55a3dcea7aacd4b72b4d, server=jenkins-hbase4.apache.org,33581,1683197625489}] 2023-05-04 10:55:36,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:36,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e26c215b73ee55a3dcea7aacd4b72b4d, NAME => 'testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d.', STARTKEY => '', ENDKEY => ''} 2023-05-04 10:55:36,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:55:36,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:36,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:55:36,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:36,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:36,225 INFO [StoreOpener-e26c215b73ee55a3dcea7aacd4b72b4d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:36,226 DEBUG [StoreOpener-e26c215b73ee55a3dcea7aacd4b72b4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1 2023-05-04 10:55:36,226 DEBUG [StoreOpener-e26c215b73ee55a3dcea7aacd4b72b4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1 2023-05-04 10:55:36,226 INFO [StoreOpener-e26c215b73ee55a3dcea7aacd4b72b4d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e26c215b73ee55a3dcea7aacd4b72b4d columnFamilyName cf1 2023-05-04 10:55:36,227 INFO [StoreOpener-e26c215b73ee55a3dcea7aacd4b72b4d-1] regionserver.HStore(310): Store=e26c215b73ee55a3dcea7aacd4b72b4d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:36,227 INFO [StoreOpener-e26c215b73ee55a3dcea7aacd4b72b4d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:36,228 DEBUG [StoreOpener-e26c215b73ee55a3dcea7aacd4b72b4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2 2023-05-04 10:55:36,228 DEBUG [StoreOpener-e26c215b73ee55a3dcea7aacd4b72b4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2 2023-05-04 10:55:36,228 INFO [StoreOpener-e26c215b73ee55a3dcea7aacd4b72b4d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e26c215b73ee55a3dcea7aacd4b72b4d columnFamilyName cf2 2023-05-04 10:55:36,229 INFO [StoreOpener-e26c215b73ee55a3dcea7aacd4b72b4d-1] regionserver.HStore(310): Store=e26c215b73ee55a3dcea7aacd4b72b4d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:36,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:36,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:36,231 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:55:36,232 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:36,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-04 10:55:36,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e26c215b73ee55a3dcea7aacd4b72b4d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58894480, jitterRate=-0.12240386009216309}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:55:36,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e26c215b73ee55a3dcea7aacd4b72b4d: 2023-05-04 10:55:36,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d., pid=143, masterSystemTime=1683197736218 2023-05-04 10:55:36,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:36,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:36,236 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=e26c215b73ee55a3dcea7aacd4b72b4d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,236 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683197736236"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197736236"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197736236"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197736236"}]},"ts":"1683197736236"} 2023-05-04 10:55:36,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-05-04 10:55:36,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure e26c215b73ee55a3dcea7aacd4b72b4d, server=jenkins-hbase4.apache.org,33581,1683197625489 in 170 msec 2023-05-04 10:55:36,240 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-05-04 10:55:36,240 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=e26c215b73ee55a3dcea7aacd4b72b4d, ASSIGN in 326 msec 2023-05-04 10:55:36,240 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-04 10:55:36,241 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197736240"}]},"ts":"1683197736240"} 2023-05-04 10:55:36,241 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-05-04 10:55:36,244 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-05-04 10:55:36,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 365 msec 2023-05-04 10:55:36,386 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:36,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-04 10:55:36,388 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-04 10:55:36,388 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:36,388 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-04 10:55:36,388 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-04 10:55:36,388 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,388 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,389 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,389 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,389 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,390 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,390 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,390 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:36,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,390 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,391 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:36,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,391 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-04 10:55:36,391 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:36,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,391 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-04 10:55:36,391 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-04 10:55:36,391 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-04 10:55:36,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:36,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,391 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:36,391 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-04 10:55:36,391 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-04 10:55:36,391 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-04 10:55:36,391 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-04 10:55:36,391 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-04 10:55:36,391 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:36,391 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-04 10:55:36,391 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-04 10:55:36,391 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-04 10:55:36,392 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-04 10:55:36,392 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:36,392 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:36,392 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:36,392 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:36,392 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-04 10:55:36,392 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-04 10:55:36,392 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-04 10:55:36,392 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. started... 2023-05-04 10:55:36,393 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e26c215b73ee55a3dcea7aacd4b72b4d 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-04 10:55:36,393 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,394 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,394 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:36,394 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,394 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,394 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:36,406 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf1/3d92eff7adab446b9d05034b61ba678f 2023-05-04 10:55:36,426 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf2/63363cd43861466cb694cef34bc9a589 2023-05-04 10:55:36,430 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf1/3d92eff7adab446b9d05034b61ba678f as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/3d92eff7adab446b9d05034b61ba678f 2023-05-04 10:55:36,433 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/3d92eff7adab446b9d05034b61ba678f, entries=2, sequenceid=6, filesize=4.8 K 2023-05-04 10:55:36,433 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf2/63363cd43861466cb694cef34bc9a589 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/63363cd43861466cb694cef34bc9a589 2023-05-04 10:55:36,436 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/63363cd43861466cb694cef34bc9a589, entries=2, sequenceid=6, filesize=4.8 K 2023-05-04 10:55:36,437 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for e26c215b73ee55a3dcea7aacd4b72b4d in 44ms, sequenceid=6, compaction requested=false 2023-05-04 10:55:36,437 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-05-04 10:55:36,437 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e26c215b73ee55a3dcea7aacd4b72b4d: 2023-05-04 10:55:36,437 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:36,437 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:36,437 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:36,438 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,438 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-04 10:55:36,438 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-04 10:55:36,440 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,440 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:36,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:36,440 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,440 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:36,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:36,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:36,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:36,442 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-04 10:55:36,442 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a2b61ca[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:36,442 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-04 10:55:36,442 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,443 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,443 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,443 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,443 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,444 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-04 10:55:36,444 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-04 10:55:36,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,444 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:36,444 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:36,443 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-04 10:55:36,444 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-04 10:55:36,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,444 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:36,444 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-04 10:55:36,444 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-04 10:55:36,444 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:36,445 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-04 10:55:36,445 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:36,445 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-04 10:55:36,445 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-04 10:55:36,446 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:36,446 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-04 10:55:36,449 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,449 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-04 10:55:36,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,449 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:36,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:36,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:36,449 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-04 10:55:36,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:36,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:36,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:36,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33581,1683197625489': 2023-05-04 10:55:36,453 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-04 10:55:36,453 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-04 10:55:36,453 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:36,453 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:36,453 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-04 10:55:36,453 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:36,454 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,454 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,454 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:36,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,454 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,454 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,454 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,455 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:36,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,455 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:36,455 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,456 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:36,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,457 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:36,457 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,458 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,460 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,460 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,460 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,460 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,460 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,460 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,460 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,460 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-04 10:55:36,460 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:36,460 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:36,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,460 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,461 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,460 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,461 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:36,461 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,461 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:36,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,461 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,461 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,461 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,461 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,461 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,461 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,561 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:36,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:36,572 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:36,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:36,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-04 10:55:36,573 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-04 10:55:36,573 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:36,573 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-04 10:55:36,573 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-04 10:55:36,574 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,574 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,576 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,576 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,576 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,576 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,577 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,577 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:36,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:36,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:36,578 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-04 10:55:36,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:36,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,578 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-04 10:55:36,578 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-04 10:55:36,578 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-04 10:55:36,578 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-04 10:55:36,578 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-04 10:55:36,578 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-04 10:55:36,578 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:36,578 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-04 10:55:36,578 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-04 10:55:36,578 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-04 10:55:36,578 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:36,578 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-04 10:55:36,578 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-04 10:55:36,579 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-04 10:55:36,579 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-04 10:55:36,579 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-04 10:55:36,580 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:36,580 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-04 10:55:36,580 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,580 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:36,580 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:36,580 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. started... 2023-05-04 10:55:36,580 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:36,580 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-04 10:55:36,580 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-04 10:55:36,580 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e26c215b73ee55a3dcea7aacd4b72b4d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:36,580 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-04 10:55:36,580 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,580 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:36,581 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,582 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,582 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:36,590 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf1/aa00a5659fb84f918e812bd1ba4d2d80 2023-05-04 10:55:36,601 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf2/5b7ca26778fb478c9962bb3b6bb378ee 2023-05-04 10:55:36,605 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf1/aa00a5659fb84f918e812bd1ba4d2d80 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/aa00a5659fb84f918e812bd1ba4d2d80 2023-05-04 10:55:36,608 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/aa00a5659fb84f918e812bd1ba4d2d80, entries=10, sequenceid=19, filesize=5.0 K 2023-05-04 10:55:36,608 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf2/5b7ca26778fb478c9962bb3b6bb378ee as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/5b7ca26778fb478c9962bb3b6bb378ee 2023-05-04 10:55:36,611 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/5b7ca26778fb478c9962bb3b6bb378ee, entries=10, sequenceid=19, filesize=5.0 K 2023-05-04 10:55:36,612 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e26c215b73ee55a3dcea7aacd4b72b4d in 32ms, sequenceid=19, compaction requested=false 2023-05-04 10:55:36,612 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e26c215b73ee55a3dcea7aacd4b72b4d: 2023-05-04 10:55:36,612 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:36,612 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:36,612 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:36,612 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,613 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-04 10:55:36,613 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-04 10:55:36,614 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,614 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:36,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:36,614 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,614 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:36,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:36,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:36,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:36,616 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-04 10:55:36,616 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7153b175[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:36,616 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-04 10:55:36,616 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,619 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,619 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,619 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,619 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,619 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-04 10:55:36,619 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-04 10:55:36,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,619 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:36,619 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-04 10:55:36,619 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-04 10:55:36,619 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:36,619 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:36,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,619 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-04 10:55:36,619 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-04 10:55:36,619 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:36,620 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-04 10:55:36,620 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:36,621 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-04 10:55:36,621 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-04 10:55:36,621 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-04 10:55:36,621 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:36,621 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:36,621 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-04 10:55:36,621 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,621 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-04 10:55:36,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:36,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:36,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:36,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:36,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:36,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33581,1683197625489': 2023-05-04 10:55:36,625 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-04 10:55:36,625 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-04 10:55:36,625 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:36,625 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:36,625 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-04 10:55:36,625 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:36,626 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,626 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,626 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,626 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:36,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:36,626 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,626 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,627 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,626 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:36,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,627 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,627 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:36,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:36,629 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,630 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,630 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,633 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,633 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,633 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,633 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,633 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:36,633 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,633 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,633 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,633 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-04 10:55:36,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,633 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,633 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,633 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:36,633 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,634 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,634 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,634 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:36,634 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,634 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,634 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,634 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:36,634 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,634 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,734 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:36,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:36,746 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:36,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:36,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-04 10:55:36,748 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-04 10:55:36,748 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:36,748 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-04 10:55:36,748 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-04 10:55:36,748 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,748 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,749 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,749 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,749 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,749 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,750 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,750 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:36,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,750 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,750 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,750 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:36,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:36,751 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-04 10:55:36,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:36,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,751 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-04 10:55:36,751 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-04 10:55:36,751 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-04 10:55:36,751 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-04 10:55:36,751 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-04 10:55:36,751 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-04 10:55:36,751 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-04 10:55:36,751 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-04 10:55:36,751 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-04 10:55:36,751 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-04 10:55:36,751 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:36,751 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-04 10:55:36,751 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-04 10:55:36,751 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:36,751 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:36,751 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-04 10:55:36,751 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-04 10:55:36,751 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-04 10:55:36,751 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-04 10:55:36,751 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-04 10:55:36,752 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-04 10:55:36,752 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:36,752 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:36,752 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:36,752 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. started... 2023-05-04 10:55:36,752 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e26c215b73ee55a3dcea7aacd4b72b4d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:36,755 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,755 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,755 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,755 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,755 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:36,755 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:36,761 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf1/8f46f539a313482a889adcf3d4fc4218 2023-05-04 10:55:36,771 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf2/28aa2ae9312b43df9f735bf4fe612bdf 2023-05-04 10:55:36,774 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf1/8f46f539a313482a889adcf3d4fc4218 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/8f46f539a313482a889adcf3d4fc4218 2023-05-04 10:55:36,778 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/8f46f539a313482a889adcf3d4fc4218, entries=10, sequenceid=32, filesize=5.0 K 2023-05-04 10:55:36,778 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf2/28aa2ae9312b43df9f735bf4fe612bdf as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/28aa2ae9312b43df9f735bf4fe612bdf 2023-05-04 10:55:36,781 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/28aa2ae9312b43df9f735bf4fe612bdf, entries=10, sequenceid=32, filesize=5.0 K 2023-05-04 10:55:36,782 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e26c215b73ee55a3dcea7aacd4b72b4d in 30ms, sequenceid=32, compaction requested=true 2023-05-04 10:55:36,782 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e26c215b73ee55a3dcea7aacd4b72b4d: 2023-05-04 10:55:36,782 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:36,782 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:36,782 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:36,782 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,782 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-04 10:55:36,782 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-04 10:55:36,784 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,784 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:36,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:36,784 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,784 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:36,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:36,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:36,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:36,786 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-04 10:55:36,786 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69c10fda[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:36,786 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-04 10:55:36,786 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,787 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,787 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,787 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,787 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-04 10:55:36,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,787 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-04 10:55:36,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,787 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:36,787 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-04 10:55:36,787 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-04 10:55:36,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,787 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:36,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,787 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,787 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-04 10:55:36,787 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-04 10:55:36,787 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:36,787 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:36,789 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-04 10:55:36,789 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,789 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:36,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:36,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:36,789 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-04 10:55:36,789 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-04 10:55:36,789 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:36,789 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-04 10:55:36,790 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-04 10:55:36,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:36,790 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:36,790 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-04 10:55:36,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:36,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:36,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33581,1683197625489': 2023-05-04 10:55:36,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-04 10:55:36,793 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-04 10:55:36,793 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:36,793 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:36,793 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-04 10:55:36,793 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:36,794 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,794 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,794 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,794 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,794 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,794 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,794 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,795 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:36,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:36,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,795 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:36,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,795 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:36,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,797 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,797 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:36,797 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,801 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,801 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,801 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,801 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,801 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,801 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,801 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,801 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-04 10:55:36,801 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:36,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,801 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,801 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:36,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,801 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:36,801 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,801 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:36,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,801 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,802 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,802 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,802 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,802 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,802 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,802 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,802 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,901 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:36,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:36,913 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:36,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:36,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-04 10:55:36,914 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-04 10:55:36,914 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:36,914 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-04 10:55:36,914 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-04 10:55:36,915 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,915 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,916 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,916 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,916 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,916 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,916 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,916 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:36,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,917 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,917 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,917 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:36,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,917 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-04 10:55:36,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:36,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:36,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,917 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-04 10:55:36,917 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-04 10:55:36,917 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-04 10:55:36,917 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-04 10:55:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-04 10:55:36,918 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-04 10:55:36,918 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-04 10:55:36,918 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-04 10:55:36,918 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-04 10:55:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-04 10:55:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-04 10:55:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-04 10:55:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-04 10:55:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-04 10:55:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-04 10:55:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-04 10:55:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-04 10:55:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-04 10:55:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:36,918 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:36,918 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. started... 2023-05-04 10:55:36,918 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e26c215b73ee55a3dcea7aacd4b72b4d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:36,919 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,920 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,920 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:36,920 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,920 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,920 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:36,931 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf1/74593406971a43e7894009bfef2f5b23 2023-05-04 10:55:36,942 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf2/24a7b7672f354946843d3d00c4fec95e 2023-05-04 10:55:36,945 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf1/74593406971a43e7894009bfef2f5b23 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/74593406971a43e7894009bfef2f5b23 2023-05-04 10:55:36,949 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/74593406971a43e7894009bfef2f5b23, entries=10, sequenceid=45, filesize=5.0 K 2023-05-04 10:55:36,949 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf2/24a7b7672f354946843d3d00c4fec95e as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/24a7b7672f354946843d3d00c4fec95e 2023-05-04 10:55:36,952 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/24a7b7672f354946843d3d00c4fec95e, entries=10, sequenceid=45, filesize=5.0 K 2023-05-04 10:55:36,953 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e26c215b73ee55a3dcea7aacd4b72b4d in 35ms, sequenceid=45, compaction requested=true 2023-05-04 10:55:36,953 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e26c215b73ee55a3dcea7aacd4b72b4d: 2023-05-04 10:55:36,953 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:36,953 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:36,953 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:36,953 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,953 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-04 10:55:36,953 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-04 10:55:36,956 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,956 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:36,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:36,956 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,956 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:36,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:36,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:36,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:36,958 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-04 10:55:36,958 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15972689[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:36,958 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-04 10:55:36,958 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,959 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,959 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,959 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,959 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-04 10:55:36,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,959 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-04 10:55:36,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,959 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,959 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:36,959 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-04 10:55:36,959 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-04 10:55:36,959 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:36,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,959 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:36,959 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-04 10:55:36,959 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-04 10:55:36,959 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:36,961 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-04 10:55:36,961 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-04 10:55:36,961 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-04 10:55:36,961 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:36,961 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,961 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-04 10:55:36,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,961 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:36,961 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:36,961 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-04 10:55:36,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:36,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:36,961 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-04 10:55:36,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:36,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:36,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:36,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33581,1683197625489': 2023-05-04 10:55:36,964 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-04 10:55:36,964 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-04 10:55:36,964 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:36,964 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:36,964 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-04 10:55:36,964 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:36,965 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,965 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,965 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,965 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,965 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,965 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,965 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,966 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:36,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:36,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,966 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:36,967 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:36,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,969 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,969 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:36,969 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:36,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:36,975 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,975 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,975 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,975 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,975 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:36,975 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegion because node does not exist (not an error) 2023-05-04 10:55:36,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,975 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-04 10:55:36,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,975 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,975 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:36,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,975 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,975 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,975 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:36,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:36,975 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:36,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:36,975 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,976 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:36,976 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:36,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,976 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:36,976 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:36,976 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:36,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:36,976 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:36,976 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:36,976 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:36,976 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,076 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:37,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:37,088 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:37,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:37,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-04 10:55:37,089 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-04 10:55:37,089 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:37,089 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-04 10:55:37,089 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-04 10:55:37,090 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,090 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:37,091 ERROR [BP-2139253951-172.31.14.131-1683197620399 heartbeating to localhost/127.0.0.1:44245] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2139253951-172.31.14.131-1683197620399 (Datanode Uuid eabd0028-d500-4e17-b99c-cf8614a5876b) service to localhost/127.0.0.1:44245 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-05-04 10:55:37,091 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:37,091 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:37,091 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:37,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:37,091 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:37,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:37,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:37,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:37,092 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,092 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:37,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:37,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:37,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:37,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:37,092 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,092 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,092 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:37,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:37,093 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-04 10:55:37,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:37,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:37,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:37,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:37,093 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-04 10:55:37,093 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-04 10:55:37,093 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-04 10:55:37,093 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-04 10:55:37,093 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-04 10:55:37,093 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:37,093 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-04 10:55:37,093 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-04 10:55:37,093 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-04 10:55:37,093 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-04 10:55:37,093 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-04 10:55:37,093 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-04 10:55:37,093 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-04 10:55:37,093 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:37,093 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-04 10:55:37,093 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-04 10:55:37,093 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-04 10:55:37,093 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-04 10:55:37,094 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-04 10:55:37,093 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:37,094 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-04 10:55:37,094 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:37,094 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:37,094 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:37,094 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. started... 2023-05-04 10:55:37,094 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e26c215b73ee55a3dcea7aacd4b72b4d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:37,094 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,095 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,095 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,095 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:37,095 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,095 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:37,103 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf1/2be3e5459a834593b2688f46009374bd 2023-05-04 10:55:37,114 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf2/2850500f7dfe4631a7c5ba3dfd40fdba 2023-05-04 10:55:37,117 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf1/2be3e5459a834593b2688f46009374bd as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/2be3e5459a834593b2688f46009374bd 2023-05-04 10:55:37,121 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/2be3e5459a834593b2688f46009374bd, entries=10, sequenceid=58, filesize=5.0 K 2023-05-04 10:55:37,121 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf2/2850500f7dfe4631a7c5ba3dfd40fdba as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/2850500f7dfe4631a7c5ba3dfd40fdba 2023-05-04 10:55:37,125 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/2850500f7dfe4631a7c5ba3dfd40fdba, entries=10, sequenceid=58, filesize=5.0 K 2023-05-04 10:55:37,125 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e26c215b73ee55a3dcea7aacd4b72b4d in 31ms, sequenceid=58, compaction requested=true 2023-05-04 10:55:37,125 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e26c215b73ee55a3dcea7aacd4b72b4d: 2023-05-04 10:55:37,125 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:37,125 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:37,125 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:37,126 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,126 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-04 10:55:37,126 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-04 10:55:37,128 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,128 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:37,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:37,128 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,128 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:37,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:37,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:37,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:37,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:37,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:37,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:37,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-04 10:55:37,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@39f98c2f[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:37,130 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-04 10:55:37,130 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,131 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,131 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,131 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,131 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-04 10:55:37,131 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-04 10:55:37,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,131 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:37,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,131 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-04 10:55:37,131 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-04 10:55:37,131 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-04 10:55:37,131 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,131 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:37,131 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:37,131 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-04 10:55:37,132 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:37,132 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-04 10:55:37,132 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,132 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:37,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:37,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:37,132 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-04 10:55:37,133 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-04 10:55:37,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:37,133 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:37,133 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-04 10:55:37,133 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:37,133 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-04 10:55:37,133 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-04 10:55:37,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:37,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:37,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:37,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:37,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:37,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:37,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:37,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:37,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33581,1683197625489': 2023-05-04 10:55:37,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-04 10:55:37,136 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-04 10:55:37,136 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:37,136 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:37,136 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-04 10:55:37,136 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:37,138 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,138 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:37,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:37,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,138 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:37,138 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,138 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:37,139 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:37,139 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:37,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,139 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:37,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:37,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:37,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:37,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:37,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:37,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:37,139 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:37,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:37,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,140 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:37,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:37,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:37,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:37,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:37,142 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:37,142 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:37,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,142 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:37,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:37,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:37,145 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:37,145 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:37,145 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:37,145 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:37,145 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:37,145 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-04 10:55:37,145 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:37,145 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:37,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:37,145 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:37,145 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:37,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:37,145 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,145 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:37,145 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:37,145 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:37,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:37,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:37,145 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:37,145 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:37,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:37,146 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:37,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:37,146 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,146 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:37,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:37,146 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:37,146 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:37,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:37,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:37,146 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:37,245 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:37,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:37,258 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-04 10:55:37,259 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-04 10:55:37,259 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-04 10:55:37,259 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-04 10:55:37,260 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:37,260 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-04 10:55:37,260 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-04 10:55:37,260 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,260 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:37,261 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:37,261 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:37,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:37,261 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,261 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-04 10:55:37,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:37,262 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:37,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:37,262 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-04 10:55:37,262 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-04 10:55:37,262 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-04 10:55:37,262 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-04 10:55:37,262 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:37,262 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-04 10:55:37,262 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-04 10:55:37,262 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37343,1683197625438' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-04 10:55:37,263 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:37,263 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:37,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:37,263 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:37,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:37,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:37,264 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,264 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:37,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:37,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:37,264 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,265 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:37,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:37,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-04 10:55:37,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:37,265 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-04 10:55:37,265 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-04 10:55:37,265 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-04 10:55:37,265 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-04 10:55:37,265 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-04 10:55:37,265 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-04 10:55:37,265 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-04 10:55:37,266 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-04 10:55:37,266 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:37,266 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-04 10:55:37,266 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-04 10:55:37,266 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-04 10:55:37,266 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-04 10:55:37,266 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42285,1683197625528' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-04 10:55:37,266 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-04 10:55:37,266 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-04 10:55:37,266 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:37,266 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. started... 2023-05-04 10:55:37,266 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e26c215b73ee55a3dcea7aacd4b72b4d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-04 10:55:37,268 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,268 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,268 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:37,282 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf1/06c6b19fd1324ae7a223e78ba7491cae 2023-05-04 10:55:37,298 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf2/496cb27b19004caf8819adff10502a03 2023-05-04 10:55:37,302 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf1/06c6b19fd1324ae7a223e78ba7491cae as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/06c6b19fd1324ae7a223e78ba7491cae 2023-05-04 10:55:37,305 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/06c6b19fd1324ae7a223e78ba7491cae, entries=10, sequenceid=71, filesize=5.0 K 2023-05-04 10:55:37,306 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf2/496cb27b19004caf8819adff10502a03 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/496cb27b19004caf8819adff10502a03 2023-05-04 10:55:37,309 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/496cb27b19004caf8819adff10502a03, entries=10, sequenceid=71, filesize=5.0 K 2023-05-04 10:55:37,310 INFO [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e26c215b73ee55a3dcea7aacd4b72b4d in 44ms, sequenceid=71, compaction requested=true 2023-05-04 10:55:37,310 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e26c215b73ee55a3dcea7aacd4b72b4d: 2023-05-04 10:55:37,310 DEBUG [rs(jenkins-hbase4.apache.org,33581,1683197625489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:37,310 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-04 10:55:37,310 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-04 10:55:37,310 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,310 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-04 10:55:37,310 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-04 10:55:37,312 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,312 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:37,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:37,313 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,313 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-04 10:55:37,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:37,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:37,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:37,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:37,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:37,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:37,314 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33581,1683197625489' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-04 10:55:37,314 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15f646f7[Count = 0] remaining members to acquire global barrier 2023-05-04 10:55:37,314 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-04 10:55:37,315 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,316 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,316 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,316 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,316 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-04 10:55:37,316 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-04 10:55:37,316 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,316 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-04 10:55:37,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,316 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,42285,1683197625528' in zk 2023-05-04 10:55:37,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,316 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-04 10:55:37,316 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-04 10:55:37,316 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37343,1683197625438' in zk 2023-05-04 10:55:37,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,316 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-04 10:55:37,317 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-04 10:55:37,317 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33581,1683197625489' in zk 2023-05-04 10:55:37,321 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-04 10:55:37,321 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:37,321 DEBUG [member: 'jenkins-hbase4.apache.org,42285,1683197625528' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-04 10:55:37,322 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-04 10:55:37,322 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,322 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:37,322 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-04 10:55:37,322 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:37,322 DEBUG [member: 'jenkins-hbase4.apache.org,33581,1683197625489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-04 10:55:37,322 DEBUG [member: 'jenkins-hbase4.apache.org,37343,1683197625438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-04 10:55:37,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:37,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:37,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:37,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:37,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:37,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:37,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:37,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:37,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:37,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:37,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:37,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33581,1683197625489': 2023-05-04 10:55:37,326 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33581,1683197625489' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-04 10:55:37,326 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-04 10:55:37,326 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-04 10:55:37,326 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-04 10:55:37,326 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-04 10:55:37,326 INFO [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-04 10:55:37,327 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,328 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,328 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,328 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:37,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-04 10:55:37,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-04 10:55:37,328 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,328 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:37,328 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:37,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:37,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:37,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:37,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:37,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:37,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-04 10:55:37,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:37,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,329 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:37,330 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:37,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-04 10:55:37,330 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:37,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:37,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:37,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:37,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-04 10:55:37,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-04 10:55:37,332 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,332 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:37,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:37,332 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:37,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:37,335 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:37,335 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:37,335 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:37,335 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:37,335 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:37,335 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:37,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:37,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:37,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:37,335 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-04 10:55:37,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:37,335 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-04 10:55:37,335 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-04 10:55:37,335 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:37,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-04 10:55:37,335 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-04 10:55:37,335 DEBUG [(jenkins-hbase4.apache.org,40233,1683197624291)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-04 10:55:37,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:37,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-04 10:55:37,335 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:37,336 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-04 10:55:37,336 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-04 10:55:37,336 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-04 10:55:37,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:37,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:37,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-04 10:55:37,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-04 10:55:37,336 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:37,336 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,336 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:37,336 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,336 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-04 10:55:37,336 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-04 10:55:37,436 DEBUG [Listener at localhost/35483] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-04 10:55:37,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-04 10:55:37,444 DEBUG [Listener at localhost/35483] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,445 DEBUG [Listener at localhost/35483] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e26c215b73ee55a3dcea7aacd4b72b4d, daughterA=c11db32a278998a4226df6ed55f7ae1b, daughterB=69937b91fcbabec4e1ff3fe143639b5c 2023-05-04 10:55:37,446 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e26c215b73ee55a3dcea7aacd4b72b4d, daughterA=c11db32a278998a4226df6ed55f7ae1b, daughterB=69937b91fcbabec4e1ff3fe143639b5c 2023-05-04 10:55:37,446 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e26c215b73ee55a3dcea7aacd4b72b4d, daughterA=c11db32a278998a4226df6ed55f7ae1b, daughterB=69937b91fcbabec4e1ff3fe143639b5c 2023-05-04 10:55:37,446 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e26c215b73ee55a3dcea7aacd4b72b4d, daughterA=c11db32a278998a4226df6ed55f7ae1b, daughterB=69937b91fcbabec4e1ff3fe143639b5c 2023-05-04 10:55:37,448 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=e26c215b73ee55a3dcea7aacd4b72b4d, UNASSIGN}] 2023-05-04 10:55:37,449 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=e26c215b73ee55a3dcea7aacd4b72b4d, UNASSIGN 2023-05-04 10:55:37,449 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=e26c215b73ee55a3dcea7aacd4b72b4d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:37,450 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683197737449"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197737449"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197737449"}]},"ts":"1683197737449"} 2023-05-04 10:55:37,450 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure e26c215b73ee55a3dcea7aacd4b72b4d, server=jenkins-hbase4.apache.org,33581,1683197625489}] 2023-05-04 10:55:37,602 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e26c215b73ee55a3dcea7aacd4b72b4d, disabling compactions & flushes 2023-05-04 10:55:37,602 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:37,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:37,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. after waiting 0 ms 2023-05-04 10:55:37,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:37,602 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing e26c215b73ee55a3dcea7aacd4b72b4d 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-04 10:55:37,612 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf1/9e3e5cc028714f15b8e81248c5cbac77 2023-05-04 10:55:37,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf2/07b7d69ecd014cbf8349af878335c25f 2023-05-04 10:55:37,626 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf1/9e3e5cc028714f15b8e81248c5cbac77 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/9e3e5cc028714f15b8e81248c5cbac77 2023-05-04 10:55:37,629 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/9e3e5cc028714f15b8e81248c5cbac77, entries=8, sequenceid=82, filesize=5.0 K 2023-05-04 10:55:37,630 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/.tmp/cf2/07b7d69ecd014cbf8349af878335c25f as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/07b7d69ecd014cbf8349af878335c25f 2023-05-04 10:55:37,633 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/07b7d69ecd014cbf8349af878335c25f, entries=8, sequenceid=82, filesize=5.0 K 2023-05-04 10:55:37,634 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for e26c215b73ee55a3dcea7aacd4b72b4d in 32ms, sequenceid=82, compaction requested=true 2023-05-04 10:55:37,639 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-04 10:55:37,639 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:55:37,639 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. 2023-05-04 10:55:37,639 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e26c215b73ee55a3dcea7aacd4b72b4d: 2023-05-04 10:55:37,640 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,641 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=e26c215b73ee55a3dcea7aacd4b72b4d, regionState=CLOSED 2023-05-04 10:55:37,641 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683197737641"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197737641"}]},"ts":"1683197737641"} 2023-05-04 10:55:37,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-05-04 10:55:37,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure e26c215b73ee55a3dcea7aacd4b72b4d, server=jenkins-hbase4.apache.org,33581,1683197625489 in 192 msec 2023-05-04 10:55:37,645 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-05-04 10:55:37,645 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=e26c215b73ee55a3dcea7aacd4b72b4d, UNASSIGN in 195 msec 2023-05-04 10:55:37,654 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=e26c215b73ee55a3dcea7aacd4b72b4d, threads=14 2023-05-04 10:55:37,655 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/07b7d69ecd014cbf8349af878335c25f for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,655 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/24a7b7672f354946843d3d00c4fec95e for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,655 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/2850500f7dfe4631a7c5ba3dfd40fdba for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,655 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/28aa2ae9312b43df9f735bf4fe612bdf for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,655 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/496cb27b19004caf8819adff10502a03 for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,657 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/5b7ca26778fb478c9962bb3b6bb378ee for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,657 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/63363cd43861466cb694cef34bc9a589 for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,657 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/06c6b19fd1324ae7a223e78ba7491cae for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,657 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/2be3e5459a834593b2688f46009374bd for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,657 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/3d92eff7adab446b9d05034b61ba678f for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,660 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/74593406971a43e7894009bfef2f5b23 for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,660 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/9e3e5cc028714f15b8e81248c5cbac77 for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,660 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/8f46f539a313482a889adcf3d4fc4218 for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,660 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/aa00a5659fb84f918e812bd1ba4d2d80 for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,776 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/06c6b19fd1324ae7a223e78ba7491cae for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,781 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/28aa2ae9312b43df9f735bf4fe612bdf for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,797 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/2be3e5459a834593b2688f46009374bd for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,825 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/24a7b7672f354946843d3d00c4fec95e for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,832 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/63363cd43861466cb694cef34bc9a589 for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,832 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/5b7ca26778fb478c9962bb3b6bb378ee for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,836 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/2850500f7dfe4631a7c5ba3dfd40fdba for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,836 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/496cb27b19004caf8819adff10502a03 for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,836 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/aa00a5659fb84f918e812bd1ba4d2d80 for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,836 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/3d92eff7adab446b9d05034b61ba678f for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:37,839 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/9e3e5cc028714f15b8e81248c5cbac77 for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:38,161 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/8f46f539a313482a889adcf3d4fc4218 for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:38,184 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/74593406971a43e7894009bfef2f5b23 for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:38,575 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/07b7d69ecd014cbf8349af878335c25f for region: e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:38,575 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region e26c215b73ee55a3dcea7aacd4b72b4d Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-04 10:55:38,614 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-04 10:55:38,615 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-04 10:55:38,617 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683197738617"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1683197738617"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1683197738617"}]},"ts":"1683197738617"} 2023-05-04 10:55:38,617 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197738617"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197738617"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197738617"}]},"ts":"1683197738617"} 2023-05-04 10:55:38,617 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197738617"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197738617"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197738617"}]},"ts":"1683197738617"} 2023-05-04 10:55:38,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=c11db32a278998a4226df6ed55f7ae1b, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=69937b91fcbabec4e1ff3fe143639b5c, ASSIGN}] 2023-05-04 10:55:38,621 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=69937b91fcbabec4e1ff3fe143639b5c, ASSIGN 2023-05-04 10:55:38,621 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=69937b91fcbabec4e1ff3fe143639b5c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33581,1683197625489; forceNewPlan=false, retain=false 2023-05-04 10:55:38,622 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=c11db32a278998a4226df6ed55f7ae1b, ASSIGN 2023-05-04 10:55:38,622 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=c11db32a278998a4226df6ed55f7ae1b, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33581,1683197625489; forceNewPlan=false, retain=false 2023-05-04 10:55:38,772 INFO [jenkins-hbase4:40233] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-04 10:55:38,773 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=c11db32a278998a4226df6ed55f7ae1b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:38,773 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197738773"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197738773"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197738773"}]},"ts":"1683197738773"} 2023-05-04 10:55:38,775 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure c11db32a278998a4226df6ed55f7ae1b, server=jenkins-hbase4.apache.org,33581,1683197625489}] 2023-05-04 10:55:38,777 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=69937b91fcbabec4e1ff3fe143639b5c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:38,777 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197738777"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197738777"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197738777"}]},"ts":"1683197738777"} 2023-05-04 10:55:38,778 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 69937b91fcbabec4e1ff3fe143639b5c, server=jenkins-hbase4.apache.org,33581,1683197625489}] 2023-05-04 10:55:38,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b. 2023-05-04 10:55:38,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c11db32a278998a4226df6ed55f7ae1b, NAME => 'testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.', STARTKEY => '', ENDKEY => '41'} 2023-05-04 10:55:38,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:55:38,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion c11db32a278998a4226df6ed55f7ae1b 2023-05-04 10:55:38,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:55:38,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c11db32a278998a4226df6ed55f7ae1b 2023-05-04 10:55:38,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c11db32a278998a4226df6ed55f7ae1b 2023-05-04 10:55:38,931 INFO [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c11db32a278998a4226df6ed55f7ae1b 2023-05-04 10:55:38,932 DEBUG [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1 2023-05-04 10:55:38,932 DEBUG [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1 2023-05-04 10:55:38,932 INFO [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c11db32a278998a4226df6ed55f7ae1b columnFamilyName cf1 2023-05-04 10:55:38,945 DEBUG [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/06c6b19fd1324ae7a223e78ba7491cae.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/06c6b19fd1324ae7a223e78ba7491cae-bottom 2023-05-04 10:55:38,948 DEBUG [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/2be3e5459a834593b2688f46009374bd.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/2be3e5459a834593b2688f46009374bd-bottom 2023-05-04 10:55:38,952 DEBUG [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/3d92eff7adab446b9d05034b61ba678f.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/3d92eff7adab446b9d05034b61ba678f-bottom 2023-05-04 10:55:38,957 DEBUG [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/74593406971a43e7894009bfef2f5b23.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/74593406971a43e7894009bfef2f5b23-bottom 2023-05-04 10:55:38,961 DEBUG [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/8f46f539a313482a889adcf3d4fc4218.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/8f46f539a313482a889adcf3d4fc4218-bottom 2023-05-04 10:55:38,964 DEBUG [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/9e3e5cc028714f15b8e81248c5cbac77.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/9e3e5cc028714f15b8e81248c5cbac77-bottom 2023-05-04 10:55:38,967 DEBUG [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/aa00a5659fb84f918e812bd1ba4d2d80.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/aa00a5659fb84f918e812bd1ba4d2d80-bottom 2023-05-04 10:55:38,968 INFO [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] regionserver.HStore(310): Store=c11db32a278998a4226df6ed55f7ae1b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:38,968 INFO [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c11db32a278998a4226df6ed55f7ae1b 2023-05-04 10:55:38,969 DEBUG [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2 2023-05-04 10:55:38,969 DEBUG [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2 2023-05-04 10:55:38,969 INFO [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c11db32a278998a4226df6ed55f7ae1b columnFamilyName cf2 2023-05-04 10:55:38,982 DEBUG [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/07b7d69ecd014cbf8349af878335c25f.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/07b7d69ecd014cbf8349af878335c25f-bottom 2023-05-04 10:55:38,985 DEBUG [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/24a7b7672f354946843d3d00c4fec95e.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/24a7b7672f354946843d3d00c4fec95e-bottom 2023-05-04 10:55:38,989 DEBUG [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/2850500f7dfe4631a7c5ba3dfd40fdba.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/2850500f7dfe4631a7c5ba3dfd40fdba-bottom 2023-05-04 10:55:38,993 DEBUG [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/28aa2ae9312b43df9f735bf4fe612bdf.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/28aa2ae9312b43df9f735bf4fe612bdf-bottom 2023-05-04 10:55:38,996 DEBUG [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/496cb27b19004caf8819adff10502a03.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/496cb27b19004caf8819adff10502a03-bottom 2023-05-04 10:55:39,000 DEBUG [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/5b7ca26778fb478c9962bb3b6bb378ee.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/5b7ca26778fb478c9962bb3b6bb378ee-bottom 2023-05-04 10:55:39,003 DEBUG [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/63363cd43861466cb694cef34bc9a589.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/63363cd43861466cb694cef34bc9a589-bottom 2023-05-04 10:55:39,004 INFO [StoreOpener-c11db32a278998a4226df6ed55f7ae1b-1] regionserver.HStore(310): Store=c11db32a278998a4226df6ed55f7ae1b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:39,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b 2023-05-04 10:55:39,005 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b 2023-05-04 10:55:39,007 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:55:39,008 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c11db32a278998a4226df6ed55f7ae1b 2023-05-04 10:55:39,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c11db32a278998a4226df6ed55f7ae1b; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63406382, jitterRate=-0.05517128109931946}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:55:39,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c11db32a278998a4226df6ed55f7ae1b: 2023-05-04 10:55:39,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b., pid=149, masterSystemTime=1683197738926 2023-05-04 10:55:39,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:55:39,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:55:39,011 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:55:39,011 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:55:39,011 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b. 2023-05-04 10:55:39,011 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b. 2023-05-04 10:55:39,011 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HStore(1912): c11db32a278998a4226df6ed55f7ae1b/cf1 is initiating minor compaction (all files) 2023-05-04 10:55:39,011 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HStore(1912): c11db32a278998a4226df6ed55f7ae1b/cf2 is initiating minor compaction (all files) 2023-05-04 10:55:39,011 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c11db32a278998a4226df6ed55f7ae1b/cf1 in testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b. 2023-05-04 10:55:39,011 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HRegion(2259): Starting compaction of c11db32a278998a4226df6ed55f7ae1b/cf2 in testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b. 2023-05-04 10:55:39,011 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/63363cd43861466cb694cef34bc9a589.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/63363cd43861466cb694cef34bc9a589-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/5b7ca26778fb478c9962bb3b6bb378ee.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/5b7ca26778fb478c9962bb3b6bb378ee-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/28aa2ae9312b43df9f735bf4fe612bdf.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/28aa2ae9312b43df9f735bf4fe612bdf-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/24a7b7672f354946843d3d00c4fec95e.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/24a7b7672f354946843d3d00c4fec95e-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/2850500f7dfe4631a7c5ba3dfd40fdba.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/2850500f7dfe4631a7c5ba3dfd40fdba-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/496cb27b19004caf8819adff10502a03.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/496cb27b19004caf8819adff10502a03-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/07b7d69ecd014cbf8349af878335c25f.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/07b7d69ecd014cbf8349af878335c25f-bottom] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/.tmp, totalSize=34.9 K 2023-05-04 10:55:39,011 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/3d92eff7adab446b9d05034b61ba678f.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/3d92eff7adab446b9d05034b61ba678f-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/aa00a5659fb84f918e812bd1ba4d2d80.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/aa00a5659fb84f918e812bd1ba4d2d80-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/8f46f539a313482a889adcf3d4fc4218.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/8f46f539a313482a889adcf3d4fc4218-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/74593406971a43e7894009bfef2f5b23.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/74593406971a43e7894009bfef2f5b23-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/2be3e5459a834593b2688f46009374bd.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/2be3e5459a834593b2688f46009374bd-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/06c6b19fd1324ae7a223e78ba7491cae.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/06c6b19fd1324ae7a223e78ba7491cae-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/9e3e5cc028714f15b8e81248c5cbac77.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/9e3e5cc028714f15b8e81248c5cbac77-bottom] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/.tmp, totalSize=34.9 K 2023-05-04 10:55:39,012 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting 63363cd43861466cb694cef34bc9a589.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683197736382 2023-05-04 10:55:39,012 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting 3d92eff7adab446b9d05034b61ba678f.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683197736382 2023-05-04 10:55:39,013 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=c11db32a278998a4226df6ed55f7ae1b, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:39,013 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197739013"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197739013"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197739013"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197739013"}]},"ts":"1683197739013"} 2023-05-04 10:55:39,015 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-05-04 10:55:39,015 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure c11db32a278998a4226df6ed55f7ae1b, server=jenkins-hbase4.apache.org,33581,1683197625489 in 239 msec 2023-05-04 10:55:39,015 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting 5b7ca26778fb478c9962bb3b6bb378ee.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683197736563 2023-05-04 10:55:39,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b. 2023-05-04 10:55:39,016 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting aa00a5659fb84f918e812bd1ba4d2d80.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683197736563 2023-05-04 10:55:39,016 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting 28aa2ae9312b43df9f735bf4fe612bdf.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683197736736 2023-05-04 10:55:39,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b. 2023-05-04 10:55:39,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c. 2023-05-04 10:55:39,016 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting 8f46f539a313482a889adcf3d4fc4218.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683197736736 2023-05-04 10:55:39,016 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=c11db32a278998a4226df6ed55f7ae1b, ASSIGN in 395 msec 2023-05-04 10:55:39,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 69937b91fcbabec4e1ff3fe143639b5c, NAME => 'testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.', STARTKEY => '41', ENDKEY => ''} 2023-05-04 10:55:39,016 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting 24a7b7672f354946843d3d00c4fec95e.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683197736903 2023-05-04 10:55:39,017 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting 74593406971a43e7894009bfef2f5b23.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683197736903 2023-05-04 10:55:39,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-04 10:55:39,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 69937b91fcbabec4e1ff3fe143639b5c 2023-05-04 10:55:39,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-04 10:55:39,017 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting 2850500f7dfe4631a7c5ba3dfd40fdba.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683197737078 2023-05-04 10:55:39,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 69937b91fcbabec4e1ff3fe143639b5c 2023-05-04 10:55:39,017 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting 2be3e5459a834593b2688f46009374bd.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683197737078 2023-05-04 10:55:39,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 69937b91fcbabec4e1ff3fe143639b5c 2023-05-04 10:55:39,018 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting 496cb27b19004caf8819adff10502a03.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683197737247 2023-05-04 10:55:39,018 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting 06c6b19fd1324ae7a223e78ba7491cae.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683197737247 2023-05-04 10:55:39,018 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting 07b7d69ecd014cbf8349af878335c25f.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683197737438 2023-05-04 10:55:39,019 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting 9e3e5cc028714f15b8e81248c5cbac77.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683197737438 2023-05-04 10:55:39,019 INFO [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 69937b91fcbabec4e1ff3fe143639b5c 2023-05-04 10:55:39,020 DEBUG [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1 2023-05-04 10:55:39,020 DEBUG [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1 2023-05-04 10:55:39,020 INFO [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 69937b91fcbabec4e1ff3fe143639b5c columnFamilyName cf1 2023-05-04 10:55:39,032 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] throttle.PressureAwareThroughputController(145): c11db32a278998a4226df6ed55f7ae1b#cf1#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:55:39,032 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] throttle.PressureAwareThroughputController(145): c11db32a278998a4226df6ed55f7ae1b#cf2#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:55:39,037 DEBUG [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/06c6b19fd1324ae7a223e78ba7491cae.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/06c6b19fd1324ae7a223e78ba7491cae-top 2023-05-04 10:55:39,045 DEBUG [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/2be3e5459a834593b2688f46009374bd.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/2be3e5459a834593b2688f46009374bd-top 2023-05-04 10:55:39,054 DEBUG [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/3d92eff7adab446b9d05034b61ba678f.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/3d92eff7adab446b9d05034b61ba678f-top 2023-05-04 10:55:39,057 DEBUG [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/74593406971a43e7894009bfef2f5b23.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/74593406971a43e7894009bfef2f5b23-top 2023-05-04 10:55:39,065 DEBUG [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/8f46f539a313482a889adcf3d4fc4218.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/8f46f539a313482a889adcf3d4fc4218-top 2023-05-04 10:55:39,066 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/.tmp/cf1/c896a7fde89241cabb62c7921b309f62 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/c896a7fde89241cabb62c7921b309f62 2023-05-04 10:55:39,069 DEBUG [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/9e3e5cc028714f15b8e81248c5cbac77.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/9e3e5cc028714f15b8e81248c5cbac77-top 2023-05-04 10:55:39,069 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/.tmp/cf2/9f41cf1d1bc543dd8bd723da8d69e8ce as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/9f41cf1d1bc543dd8bd723da8d69e8ce 2023-05-04 10:55:39,071 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c11db32a278998a4226df6ed55f7ae1b/cf1 of c11db32a278998a4226df6ed55f7ae1b into c896a7fde89241cabb62c7921b309f62(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:55:39,071 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c11db32a278998a4226df6ed55f7ae1b: 2023-05-04 10:55:39,071 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b., storeName=c11db32a278998a4226df6ed55f7ae1b/cf1, priority=9, startTime=1683197739010; duration=0sec 2023-05-04 10:55:39,072 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:55:39,072 DEBUG [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/aa00a5659fb84f918e812bd1ba4d2d80.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/aa00a5659fb84f918e812bd1ba4d2d80-top 2023-05-04 10:55:39,073 INFO [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] regionserver.HStore(310): Store=69937b91fcbabec4e1ff3fe143639b5c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:39,073 INFO [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 69937b91fcbabec4e1ff3fe143639b5c 2023-05-04 10:55:39,074 DEBUG [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2 2023-05-04 10:55:39,074 DEBUG [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2 2023-05-04 10:55:39,074 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c11db32a278998a4226df6ed55f7ae1b/cf2 of c11db32a278998a4226df6ed55f7ae1b into 9f41cf1d1bc543dd8bd723da8d69e8ce(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-04 10:55:39,074 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for c11db32a278998a4226df6ed55f7ae1b: 2023-05-04 10:55:39,074 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b., storeName=c11db32a278998a4226df6ed55f7ae1b/cf2, priority=9, startTime=1683197739010; duration=0sec 2023-05-04 10:55:39,074 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:55:39,074 INFO [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 69937b91fcbabec4e1ff3fe143639b5c columnFamilyName cf2 2023-05-04 10:55:39,086 DEBUG [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/07b7d69ecd014cbf8349af878335c25f.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/07b7d69ecd014cbf8349af878335c25f-top 2023-05-04 10:55:39,089 DEBUG [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/24a7b7672f354946843d3d00c4fec95e.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/24a7b7672f354946843d3d00c4fec95e-top 2023-05-04 10:55:39,092 DEBUG [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/2850500f7dfe4631a7c5ba3dfd40fdba.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/2850500f7dfe4631a7c5ba3dfd40fdba-top 2023-05-04 10:55:39,092 ERROR [BP-2139253951-172.31.14.131-1683197620399 heartbeating to localhost/127.0.0.1:44245] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2139253951-172.31.14.131-1683197620399 (Datanode Uuid 4e8a05ef-1582-45ea-8d3b-86bbf8f33f91) service to localhost/127.0.0.1:44245 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-05-04 10:55:39,095 DEBUG [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/28aa2ae9312b43df9f735bf4fe612bdf.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/28aa2ae9312b43df9f735bf4fe612bdf-top 2023-05-04 10:55:39,098 DEBUG [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/496cb27b19004caf8819adff10502a03.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/496cb27b19004caf8819adff10502a03-top 2023-05-04 10:55:39,101 DEBUG [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/5b7ca26778fb478c9962bb3b6bb378ee.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/5b7ca26778fb478c9962bb3b6bb378ee-top 2023-05-04 10:55:39,104 DEBUG [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] regionserver.HStore(539): loaded hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/63363cd43861466cb694cef34bc9a589.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/63363cd43861466cb694cef34bc9a589-top 2023-05-04 10:55:39,104 INFO [StoreOpener-69937b91fcbabec4e1ff3fe143639b5c-1] regionserver.HStore(310): Store=69937b91fcbabec4e1ff3fe143639b5c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-04 10:55:39,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c 2023-05-04 10:55:39,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c 2023-05-04 10:55:39,106 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-04 10:55:39,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 69937b91fcbabec4e1ff3fe143639b5c 2023-05-04 10:55:39,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 69937b91fcbabec4e1ff3fe143639b5c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61997937, jitterRate=-0.07615874707698822}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-04 10:55:39,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 69937b91fcbabec4e1ff3fe143639b5c: 2023-05-04 10:55:39,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c., pid=150, masterSystemTime=1683197738926 2023-05-04 10:55:39,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:55:39,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:55:39,110 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:55:39,110 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-04 10:55:39,110 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c. 2023-05-04 10:55:39,110 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HStore(1912): 69937b91fcbabec4e1ff3fe143639b5c/cf2 is initiating minor compaction (all files) 2023-05-04 10:55:39,110 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c. 2023-05-04 10:55:39,110 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 69937b91fcbabec4e1ff3fe143639b5c/cf2 in testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c. 2023-05-04 10:55:39,110 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HStore(1912): 69937b91fcbabec4e1ff3fe143639b5c/cf1 is initiating minor compaction (all files) 2023-05-04 10:55:39,110 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 69937b91fcbabec4e1ff3fe143639b5c/cf1 in testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c. 2023-05-04 10:55:39,110 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/63363cd43861466cb694cef34bc9a589.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/63363cd43861466cb694cef34bc9a589-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/5b7ca26778fb478c9962bb3b6bb378ee.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/5b7ca26778fb478c9962bb3b6bb378ee-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/28aa2ae9312b43df9f735bf4fe612bdf.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/28aa2ae9312b43df9f735bf4fe612bdf-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/24a7b7672f354946843d3d00c4fec95e.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/24a7b7672f354946843d3d00c4fec95e-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/2850500f7dfe4631a7c5ba3dfd40fdba.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/2850500f7dfe4631a7c5ba3dfd40fdba-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/496cb27b19004caf8819adff10502a03.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/496cb27b19004caf8819adff10502a03-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/07b7d69ecd014cbf8349af878335c25f.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/07b7d69ecd014cbf8349af878335c25f-top] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/.tmp, totalSize=34.9 K 2023-05-04 10:55:39,148 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/3d92eff7adab446b9d05034b61ba678f.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/3d92eff7adab446b9d05034b61ba678f-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/aa00a5659fb84f918e812bd1ba4d2d80.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/aa00a5659fb84f918e812bd1ba4d2d80-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/8f46f539a313482a889adcf3d4fc4218.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/8f46f539a313482a889adcf3d4fc4218-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/74593406971a43e7894009bfef2f5b23.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/74593406971a43e7894009bfef2f5b23-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/2be3e5459a834593b2688f46009374bd.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/2be3e5459a834593b2688f46009374bd-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/06c6b19fd1324ae7a223e78ba7491cae.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/06c6b19fd1324ae7a223e78ba7491cae-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/9e3e5cc028714f15b8e81248c5cbac77.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/9e3e5cc028714f15b8e81248c5cbac77-top] into tmpdir=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/.tmp, totalSize=34.9 K 2023-05-04 10:55:39,148 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting 63363cd43861466cb694cef34bc9a589.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683197736382 2023-05-04 10:55:39,149 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting 3d92eff7adab446b9d05034b61ba678f.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683197736382 2023-05-04 10:55:39,149 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c. 2023-05-04 10:55:39,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c. 2023-05-04 10:55:39,149 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting 5b7ca26778fb478c9962bb3b6bb378ee.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683197736563 2023-05-04 10:55:39,149 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting aa00a5659fb84f918e812bd1ba4d2d80.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683197736563 2023-05-04 10:55:39,149 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting 28aa2ae9312b43df9f735bf4fe612bdf.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683197736736 2023-05-04 10:55:39,150 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting 8f46f539a313482a889adcf3d4fc4218.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683197736736 2023-05-04 10:55:39,150 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting 24a7b7672f354946843d3d00c4fec95e.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683197736903 2023-05-04 10:55:39,150 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting 74593406971a43e7894009bfef2f5b23.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683197736903 2023-05-04 10:55:39,150 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting 2850500f7dfe4631a7c5ba3dfd40fdba.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1683197737078 2023-05-04 10:55:39,150 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=69937b91fcbabec4e1ff3fe143639b5c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:39,150 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197739150"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683197739150"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683197739150"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683197739150"}]},"ts":"1683197739150"} 2023-05-04 10:55:39,150 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting 2be3e5459a834593b2688f46009374bd.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1683197737078 2023-05-04 10:55:39,151 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting 496cb27b19004caf8819adff10502a03.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1683197737247 2023-05-04 10:55:39,151 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting 06c6b19fd1324ae7a223e78ba7491cae.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1683197737247 2023-05-04 10:55:39,151 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] compactions.Compactor(207): Compacting 07b7d69ecd014cbf8349af878335c25f.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683197737438 2023-05-04 10:55:39,151 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] compactions.Compactor(207): Compacting 9e3e5cc028714f15b8e81248c5cbac77.e26c215b73ee55a3dcea7aacd4b72b4d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683197737438 2023-05-04 10:55:39,153 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-05-04 10:55:39,153 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 69937b91fcbabec4e1ff3fe143639b5c, server=jenkins-hbase4.apache.org,33581,1683197625489 in 374 msec 2023-05-04 10:55:39,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=148, resume processing ppid=144 2023-05-04 10:55:39,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=69937b91fcbabec4e1ff3fe143639b5c, ASSIGN in 533 msec 2023-05-04 10:55:39,156 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=e26c215b73ee55a3dcea7aacd4b72b4d, daughterA=c11db32a278998a4226df6ed55f7ae1b, daughterB=69937b91fcbabec4e1ff3fe143639b5c in 1.7100 sec 2023-05-04 10:55:39,161 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] throttle.PressureAwareThroughputController(145): 69937b91fcbabec4e1ff3fe143639b5c#cf1#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:55:39,161 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] throttle.PressureAwareThroughputController(145): 69937b91fcbabec4e1ff3fe143639b5c#cf2#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-04 10:55:39,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33581] ipc.CallRunner(144): callId: 1148 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:40236 deadline: 1683197799197, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d. is not online on jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:39,307 DEBUG [Listener at localhost/35483] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-04 10:55:39,311 INFO [Listener at localhost/35483] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-05-04 10:55:39,313 DEBUG [Listener at localhost/35483] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-04 10:55:39,334 WARN [Listener at localhost/35483] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-04 10:55:39,335 INFO [Listener at localhost/35483] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-05-04 10:55:39,336 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-05-04 10:55:39,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-05-04 10:55:39,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-05-04 10:55:39,339 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197739339"}]},"ts":"1683197739339"} 2023-05-04 10:55:39,342 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-05-04 10:55:39,344 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-05-04 10:55:39,345 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=c11db32a278998a4226df6ed55f7ae1b, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=69937b91fcbabec4e1ff3fe143639b5c, UNASSIGN}] 2023-05-04 10:55:39,345 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=69937b91fcbabec4e1ff3fe143639b5c, UNASSIGN 2023-05-04 10:55:39,346 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=69937b91fcbabec4e1ff3fe143639b5c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:39,346 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197739346"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197739346"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197739346"}]},"ts":"1683197739346"} 2023-05-04 10:55:39,347 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 69937b91fcbabec4e1ff3fe143639b5c, server=jenkins-hbase4.apache.org,33581,1683197625489}] 2023-05-04 10:55:39,349 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=c11db32a278998a4226df6ed55f7ae1b, UNASSIGN 2023-05-04 10:55:39,349 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=c11db32a278998a4226df6ed55f7ae1b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:39,349 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197739349"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683197739349"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683197739349"}]},"ts":"1683197739349"} 2023-05-04 10:55:39,353 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure c11db32a278998a4226df6ed55f7ae1b, server=jenkins-hbase4.apache.org,33581,1683197625489}] 2023-05-04 10:55:39,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-05-04 10:55:39,499 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close c11db32a278998a4226df6ed55f7ae1b 2023-05-04 10:55:39,499 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c11db32a278998a4226df6ed55f7ae1b, disabling compactions & flushes 2023-05-04 10:55:39,499 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b. 2023-05-04 10:55:39,499 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b. 2023-05-04 10:55:39,499 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b. after waiting 0 ms 2023-05-04 10:55:39,499 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b. 2023-05-04 10:55:39,500 DEBUG [StoreCloser-testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/3d92eff7adab446b9d05034b61ba678f.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/3d92eff7adab446b9d05034b61ba678f-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/aa00a5659fb84f918e812bd1ba4d2d80.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/aa00a5659fb84f918e812bd1ba4d2d80-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/8f46f539a313482a889adcf3d4fc4218.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/8f46f539a313482a889adcf3d4fc4218-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/74593406971a43e7894009bfef2f5b23.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/74593406971a43e7894009bfef2f5b23-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/2be3e5459a834593b2688f46009374bd.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/2be3e5459a834593b2688f46009374bd-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/06c6b19fd1324ae7a223e78ba7491cae.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/06c6b19fd1324ae7a223e78ba7491cae-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/9e3e5cc028714f15b8e81248c5cbac77.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/9e3e5cc028714f15b8e81248c5cbac77-bottom] to archive 2023-05-04 10:55:39,501 DEBUG [StoreCloser-testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:55:39,502 DEBUG [StoreCloser-testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/3d92eff7adab446b9d05034b61ba678f.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/3d92eff7adab446b9d05034b61ba678f.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:39,504 DEBUG [StoreCloser-testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/aa00a5659fb84f918e812bd1ba4d2d80.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/aa00a5659fb84f918e812bd1ba4d2d80.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:39,505 DEBUG [StoreCloser-testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/8f46f539a313482a889adcf3d4fc4218.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/8f46f539a313482a889adcf3d4fc4218.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:39,505 DEBUG [StoreCloser-testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/74593406971a43e7894009bfef2f5b23.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/74593406971a43e7894009bfef2f5b23.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:39,506 DEBUG [StoreCloser-testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/2be3e5459a834593b2688f46009374bd.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/2be3e5459a834593b2688f46009374bd.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:39,507 DEBUG [StoreCloser-testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/06c6b19fd1324ae7a223e78ba7491cae.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/06c6b19fd1324ae7a223e78ba7491cae.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:39,508 DEBUG [StoreCloser-testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/9e3e5cc028714f15b8e81248c5cbac77.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/9e3e5cc028714f15b8e81248c5cbac77.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:39,513 DEBUG [StoreCloser-testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/63363cd43861466cb694cef34bc9a589.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/63363cd43861466cb694cef34bc9a589-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/5b7ca26778fb478c9962bb3b6bb378ee.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/5b7ca26778fb478c9962bb3b6bb378ee-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/28aa2ae9312b43df9f735bf4fe612bdf.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/28aa2ae9312b43df9f735bf4fe612bdf-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/24a7b7672f354946843d3d00c4fec95e.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/24a7b7672f354946843d3d00c4fec95e-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/2850500f7dfe4631a7c5ba3dfd40fdba.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/2850500f7dfe4631a7c5ba3dfd40fdba-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/496cb27b19004caf8819adff10502a03.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/496cb27b19004caf8819adff10502a03-bottom, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/07b7d69ecd014cbf8349af878335c25f.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/07b7d69ecd014cbf8349af878335c25f-bottom] to archive 2023-05-04 10:55:39,513 DEBUG [StoreCloser-testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:55:39,514 DEBUG [StoreCloser-testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/63363cd43861466cb694cef34bc9a589.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/63363cd43861466cb694cef34bc9a589.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:39,515 DEBUG [StoreCloser-testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/5b7ca26778fb478c9962bb3b6bb378ee.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/5b7ca26778fb478c9962bb3b6bb378ee.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:39,516 DEBUG [StoreCloser-testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/28aa2ae9312b43df9f735bf4fe612bdf.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/28aa2ae9312b43df9f735bf4fe612bdf.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:39,517 DEBUG [StoreCloser-testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/24a7b7672f354946843d3d00c4fec95e.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/24a7b7672f354946843d3d00c4fec95e.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:39,518 DEBUG [StoreCloser-testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/2850500f7dfe4631a7c5ba3dfd40fdba.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/2850500f7dfe4631a7c5ba3dfd40fdba.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:39,519 DEBUG [StoreCloser-testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/496cb27b19004caf8819adff10502a03.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/496cb27b19004caf8819adff10502a03.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:39,520 DEBUG [StoreCloser-testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/07b7d69ecd014cbf8349af878335c25f.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/07b7d69ecd014cbf8349af878335c25f.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:39,523 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-04 10:55:39,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:55:39,524 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b. 2023-05-04 10:55:39,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c11db32a278998a4226df6ed55f7ae1b: 2023-05-04 10:55:39,525 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed c11db32a278998a4226df6ed55f7ae1b 2023-05-04 10:55:39,525 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 69937b91fcbabec4e1ff3fe143639b5c 2023-05-04 10:55:39,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 69937b91fcbabec4e1ff3fe143639b5c, disabling compactions & flushes 2023-05-04 10:55:39,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 2 compactions to complete for region testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c. 2023-05-04 10:55:39,525 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=c11db32a278998a4226df6ed55f7ae1b, regionState=CLOSED 2023-05-04 10:55:39,525 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197739525"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197739525"}]},"ts":"1683197739525"} 2023-05-04 10:55:39,527 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-05-04 10:55:39,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure c11db32a278998a4226df6ed55f7ae1b, server=jenkins-hbase4.apache.org,33581,1683197625489 in 176 msec 2023-05-04 10:55:39,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=c11db32a278998a4226df6ed55f7ae1b, UNASSIGN in 183 msec 2023-05-04 10:55:39,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-05-04 10:55:39,943 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-05-04 10:55:39,949 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-04 10:55:40,380 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/.tmp/cf1/65d12534051b4a66942b09abde6befa2 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/65d12534051b4a66942b09abde6befa2 2023-05-04 10:55:40,380 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/.tmp/cf2/755f59daecf5433188182ebe234e3ec6 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/755f59daecf5433188182ebe234e3ec6 2023-05-04 10:55:40,384 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 69937b91fcbabec4e1ff3fe143639b5c/cf1 of 69937b91fcbabec4e1ff3fe143639b5c into 65d12534051b4a66942b09abde6befa2(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-05-04 10:55:40,384 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 69937b91fcbabec4e1ff3fe143639b5c: 2023-05-04 10:55:40,384 INFO [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c., storeName=69937b91fcbabec4e1ff3fe143639b5c/cf1, priority=9, startTime=1683197739109; duration=1sec 2023-05-04 10:55:40,384 DEBUG [RS:1;jenkins-hbase4:33581-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:55:40,384 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 69937b91fcbabec4e1ff3fe143639b5c/cf2 of 69937b91fcbabec4e1ff3fe143639b5c into 755f59daecf5433188182ebe234e3ec6(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-05-04 10:55:40,384 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 69937b91fcbabec4e1ff3fe143639b5c: 2023-05-04 10:55:40,384 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c. 2023-05-04 10:55:40,384 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c. 2023-05-04 10:55:40,384 INFO [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c., storeName=69937b91fcbabec4e1ff3fe143639b5c/cf2, priority=9, startTime=1683197739110; duration=1sec 2023-05-04 10:55:40,384 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c. after waiting 0 ms 2023-05-04 10:55:40,384 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c. 2023-05-04 10:55:40,384 DEBUG [RS:1;jenkins-hbase4:33581-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-04 10:55:40,385 DEBUG [StoreCloser-testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/3d92eff7adab446b9d05034b61ba678f.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/3d92eff7adab446b9d05034b61ba678f-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/aa00a5659fb84f918e812bd1ba4d2d80.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/aa00a5659fb84f918e812bd1ba4d2d80-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/8f46f539a313482a889adcf3d4fc4218.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/8f46f539a313482a889adcf3d4fc4218-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/74593406971a43e7894009bfef2f5b23.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/74593406971a43e7894009bfef2f5b23-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/2be3e5459a834593b2688f46009374bd.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/2be3e5459a834593b2688f46009374bd-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/06c6b19fd1324ae7a223e78ba7491cae.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/06c6b19fd1324ae7a223e78ba7491cae-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/9e3e5cc028714f15b8e81248c5cbac77.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/9e3e5cc028714f15b8e81248c5cbac77-top] to archive 2023-05-04 10:55:40,386 DEBUG [StoreCloser-testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:55:40,387 DEBUG [StoreCloser-testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/3d92eff7adab446b9d05034b61ba678f.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/3d92eff7adab446b9d05034b61ba678f.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:40,388 DEBUG [StoreCloser-testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/aa00a5659fb84f918e812bd1ba4d2d80.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/aa00a5659fb84f918e812bd1ba4d2d80.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:40,389 DEBUG [StoreCloser-testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/8f46f539a313482a889adcf3d4fc4218.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/8f46f539a313482a889adcf3d4fc4218.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:40,390 DEBUG [StoreCloser-testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/74593406971a43e7894009bfef2f5b23.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/74593406971a43e7894009bfef2f5b23.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:40,391 DEBUG [StoreCloser-testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/2be3e5459a834593b2688f46009374bd.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/2be3e5459a834593b2688f46009374bd.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:40,392 DEBUG [StoreCloser-testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/06c6b19fd1324ae7a223e78ba7491cae.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/06c6b19fd1324ae7a223e78ba7491cae.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:40,393 DEBUG [StoreCloser-testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/9e3e5cc028714f15b8e81248c5cbac77.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/9e3e5cc028714f15b8e81248c5cbac77.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:40,395 DEBUG [StoreCloser-testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/63363cd43861466cb694cef34bc9a589.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/63363cd43861466cb694cef34bc9a589-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/5b7ca26778fb478c9962bb3b6bb378ee.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/5b7ca26778fb478c9962bb3b6bb378ee-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/28aa2ae9312b43df9f735bf4fe612bdf.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/28aa2ae9312b43df9f735bf4fe612bdf-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/24a7b7672f354946843d3d00c4fec95e.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/24a7b7672f354946843d3d00c4fec95e-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/2850500f7dfe4631a7c5ba3dfd40fdba.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/2850500f7dfe4631a7c5ba3dfd40fdba-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/496cb27b19004caf8819adff10502a03.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/496cb27b19004caf8819adff10502a03-top, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/07b7d69ecd014cbf8349af878335c25f.e26c215b73ee55a3dcea7aacd4b72b4d->hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/07b7d69ecd014cbf8349af878335c25f-top] to archive 2023-05-04 10:55:40,395 DEBUG [StoreCloser-testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-04 10:55:40,396 DEBUG [StoreCloser-testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/63363cd43861466cb694cef34bc9a589.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/63363cd43861466cb694cef34bc9a589.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:40,398 DEBUG [StoreCloser-testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/5b7ca26778fb478c9962bb3b6bb378ee.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/5b7ca26778fb478c9962bb3b6bb378ee.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:40,399 DEBUG [StoreCloser-testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/28aa2ae9312b43df9f735bf4fe612bdf.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/28aa2ae9312b43df9f735bf4fe612bdf.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:40,400 DEBUG [StoreCloser-testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/24a7b7672f354946843d3d00c4fec95e.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/24a7b7672f354946843d3d00c4fec95e.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:40,401 DEBUG [StoreCloser-testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/2850500f7dfe4631a7c5ba3dfd40fdba.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/2850500f7dfe4631a7c5ba3dfd40fdba.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:40,402 DEBUG [StoreCloser-testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/496cb27b19004caf8819adff10502a03.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/496cb27b19004caf8819adff10502a03.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:40,402 DEBUG [StoreCloser-testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/07b7d69ecd014cbf8349af878335c25f.e26c215b73ee55a3dcea7aacd4b72b4d to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/07b7d69ecd014cbf8349af878335c25f.e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:40,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-04 10:55:40,406 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:55:40,407 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c. 2023-05-04 10:55:40,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 69937b91fcbabec4e1ff3fe143639b5c: 2023-05-04 10:55:40,408 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 69937b91fcbabec4e1ff3fe143639b5c 2023-05-04 10:55:40,409 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=69937b91fcbabec4e1ff3fe143639b5c, regionState=CLOSED 2023-05-04 10:55:40,409 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683197740409"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683197740409"}]},"ts":"1683197740409"} 2023-05-04 10:55:40,411 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-05-04 10:55:40,411 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 69937b91fcbabec4e1ff3fe143639b5c, server=jenkins-hbase4.apache.org,33581,1683197625489 in 1.0630 sec 2023-05-04 10:55:40,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=153, resume processing ppid=151 2023-05-04 10:55:40,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=69937b91fcbabec4e1ff3fe143639b5c, UNASSIGN in 1.0660 sec 2023-05-04 10:55:40,413 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683197740413"}]},"ts":"1683197740413"} 2023-05-04 10:55:40,413 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-05-04 10:55:40,415 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-05-04 10:55:40,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 1.0790 sec 2023-05-04 10:55:40,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-05-04 10:55:40,444 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-05-04 10:55:40,445 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-05-04 10:55:40,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-05-04 10:55:40,447 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-04 10:55:40,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-05-04 10:55:40,447 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-04 10:55:40,452 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:40,452 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c 2023-05-04 10:55:40,452 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b 2023-05-04 10:55:40,455 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/recovered.edits] 2023-05-04 10:55:40,456 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/recovered.edits] 2023-05-04 10:55:40,457 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2, FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/recovered.edits] 2023-05-04 10:55:40,460 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/c896a7fde89241cabb62c7921b309f62 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf1/c896a7fde89241cabb62c7921b309f62 2023-05-04 10:55:40,461 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/06c6b19fd1324ae7a223e78ba7491cae to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/06c6b19fd1324ae7a223e78ba7491cae 2023-05-04 10:55:40,461 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/65d12534051b4a66942b09abde6befa2 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf1/65d12534051b4a66942b09abde6befa2 2023-05-04 10:55:40,462 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/2be3e5459a834593b2688f46009374bd to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/2be3e5459a834593b2688f46009374bd 2023-05-04 10:55:40,463 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/9f41cf1d1bc543dd8bd723da8d69e8ce to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/cf2/9f41cf1d1bc543dd8bd723da8d69e8ce 2023-05-04 10:55:40,464 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/3d92eff7adab446b9d05034b61ba678f to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/3d92eff7adab446b9d05034b61ba678f 2023-05-04 10:55:40,465 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/755f59daecf5433188182ebe234e3ec6 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/cf2/755f59daecf5433188182ebe234e3ec6 2023-05-04 10:55:40,466 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/74593406971a43e7894009bfef2f5b23 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/74593406971a43e7894009bfef2f5b23 2023-05-04 10:55:40,466 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/recovered.edits/90.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b/recovered.edits/90.seqid 2023-05-04 10:55:40,467 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/c11db32a278998a4226df6ed55f7ae1b 2023-05-04 10:55:40,467 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/8f46f539a313482a889adcf3d4fc4218 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/8f46f539a313482a889adcf3d4fc4218 2023-05-04 10:55:40,468 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/recovered.edits/90.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c/recovered.edits/90.seqid 2023-05-04 10:55:40,469 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/9e3e5cc028714f15b8e81248c5cbac77 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/9e3e5cc028714f15b8e81248c5cbac77 2023-05-04 10:55:40,469 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/69937b91fcbabec4e1ff3fe143639b5c 2023-05-04 10:55:40,470 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/aa00a5659fb84f918e812bd1ba4d2d80 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf1/aa00a5659fb84f918e812bd1ba4d2d80 2023-05-04 10:55:40,473 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/07b7d69ecd014cbf8349af878335c25f to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/07b7d69ecd014cbf8349af878335c25f 2023-05-04 10:55:40,474 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/24a7b7672f354946843d3d00c4fec95e to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/24a7b7672f354946843d3d00c4fec95e 2023-05-04 10:55:40,476 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/2850500f7dfe4631a7c5ba3dfd40fdba to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/2850500f7dfe4631a7c5ba3dfd40fdba 2023-05-04 10:55:40,477 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/28aa2ae9312b43df9f735bf4fe612bdf to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/28aa2ae9312b43df9f735bf4fe612bdf 2023-05-04 10:55:40,478 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/496cb27b19004caf8819adff10502a03 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/496cb27b19004caf8819adff10502a03 2023-05-04 10:55:40,479 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/5b7ca26778fb478c9962bb3b6bb378ee to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/5b7ca26778fb478c9962bb3b6bb378ee 2023-05-04 10:55:40,480 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/63363cd43861466cb694cef34bc9a589 to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/cf2/63363cd43861466cb694cef34bc9a589 2023-05-04 10:55:40,482 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/recovered.edits/85.seqid to hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/archive/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d/recovered.edits/85.seqid 2023-05-04 10:55:40,483 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/.tmp/data/default/testSplitTableRegion/e26c215b73ee55a3dcea7aacd4b72b4d 2023-05-04 10:55:40,483 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-05-04 10:55:40,485 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-04 10:55:40,487 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-05-04 10:55:40,488 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-05-04 10:55:40,489 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-04 10:55:40,489 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-05-04 10:55:40,489 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197740489"}]},"ts":"9223372036854775807"} 2023-05-04 10:55:40,489 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197740489"}]},"ts":"9223372036854775807"} 2023-05-04 10:55:40,489 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683197740489"}]},"ts":"9223372036854775807"} 2023-05-04 10:55:40,491 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-04 10:55:40,491 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => e26c215b73ee55a3dcea7aacd4b72b4d, NAME => 'testSplitTableRegion,,1683197735879.e26c215b73ee55a3dcea7aacd4b72b4d.', STARTKEY => '', ENDKEY => ''}, {ENCODED => c11db32a278998a4226df6ed55f7ae1b, NAME => 'testSplitTableRegion,,1683197737445.c11db32a278998a4226df6ed55f7ae1b.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 69937b91fcbabec4e1ff3fe143639b5c, NAME => 'testSplitTableRegion,41,1683197737445.69937b91fcbabec4e1ff3fe143639b5c.', STARTKEY => '41', ENDKEY => ''}] 2023-05-04 10:55:40,491 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-05-04 10:55:40,491 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683197740491"}]},"ts":"9223372036854775807"} 2023-05-04 10:55:40,493 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-05-04 10:55:40,494 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-04 10:55:40,495 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 49 msec 2023-05-04 10:55:40,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40233] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-05-04 10:55:40,549 INFO [Listener at localhost/35483] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-05-04 10:55:40,565 INFO [Listener at localhost/35483] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=376 (was 374) - Thread LEAK? -, OpenFileDescriptor=825 (was 829), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=187 (was 177) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=6293 (was 6311) 2023-05-04 10:55:40,565 INFO [Listener at localhost/35483] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-05-04 10:55:40,566 INFO [Listener at localhost/35483] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-05-04 10:55:40,566 DEBUG [Listener at localhost/35483] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7b7d11f1 to 127.0.0.1:53687 2023-05-04 10:55:40,566 DEBUG [Listener at localhost/35483] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-04 10:55:40,567 DEBUG [Listener at localhost/35483] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-05-04 10:55:40,567 DEBUG [Listener at localhost/35483] util.JVMClusterUtil(257): Found active master hash=126844141, stopped=false 2023-05-04 10:55:40,567 INFO [Listener at localhost/35483] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,40233,1683197624291 2023-05-04 10:55:40,568 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-05-04 10:55:40,568 INFO [Listener at localhost/35483] procedure2.ProcedureExecutor(629): Stopping 2023-05-04 10:55:40,568 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-05-04 10:55:40,568 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-05-04 10:55:40,568 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-04 10:55:40,569 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-04 10:55:40,569 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-05-04 10:55:40,569 DEBUG [Listener at localhost/35483] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2fa7d594 to 127.0.0.1:53687 2023-05-04 10:55:40,569 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-04 10:55:40,569 DEBUG [Listener at localhost/35483] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-04 10:55:40,569 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-04 10:55:40,570 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-04 10:55:40,570 INFO [Listener at localhost/35483] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,37343,1683197625438' ***** 2023-05-04 10:55:40,570 INFO [Listener at localhost/35483] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-05-04 10:55:40,570 INFO [RS:0;jenkins-hbase4:37343] regionserver.HeapMemoryManager(220): Stopping 2023-05-04 10:55:40,571 INFO [Listener at localhost/35483] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,33581,1683197625489' ***** 2023-05-04 10:55:40,571 INFO [Listener at localhost/35483] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-05-04 10:55:40,571 INFO [RS:0;jenkins-hbase4:37343] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-05-04 10:55:40,571 INFO [RS:1;jenkins-hbase4:33581] regionserver.HeapMemoryManager(220): Stopping 2023-05-04 10:55:40,571 INFO [RS:0;jenkins-hbase4:37343] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-05-04 10:55:40,571 INFO [RS:1;jenkins-hbase4:33581] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-05-04 10:55:40,571 INFO [RS:1;jenkins-hbase4:33581] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-05-04 10:55:40,571 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-05-04 10:55:40,571 INFO [RS:0;jenkins-hbase4:37343] regionserver.HRegionServer(3304): Received CLOSE for 468c84c2202e377e79810bff249e3ae7 2023-05-04 10:55:40,571 INFO [Listener at localhost/35483] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,42285,1683197625528' ***** 2023-05-04 10:55:40,571 INFO [RS:1;jenkins-hbase4:33581] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:40,571 DEBUG [RS:1;jenkins-hbase4:33581] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1cabd420 to 127.0.0.1:53687 2023-05-04 10:55:40,572 INFO [RS:0;jenkins-hbase4:37343] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:40,572 DEBUG [RS:0;jenkins-hbase4:37343] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x66f3f124 to 127.0.0.1:53687 2023-05-04 10:55:40,572 DEBUG [RS:0;jenkins-hbase4:37343] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-04 10:55:40,572 DEBUG [RS:1;jenkins-hbase4:33581] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-04 10:55:40,572 INFO [RS:0;jenkins-hbase4:37343] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-05-04 10:55:40,572 INFO [RS:1;jenkins-hbase4:33581] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,33581,1683197625489; all regions closed. 2023-05-04 10:55:40,572 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-05-04 10:55:40,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 468c84c2202e377e79810bff249e3ae7, disabling compactions & flushes 2023-05-04 10:55:40,572 INFO [Listener at localhost/35483] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-05-04 10:55:40,572 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7. 2023-05-04 10:55:40,572 DEBUG [RS:0;jenkins-hbase4:37343] regionserver.HRegionServer(1479): Online Regions={468c84c2202e377e79810bff249e3ae7=hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7.} 2023-05-04 10:55:40,572 INFO [RS:2;jenkins-hbase4:42285] regionserver.HeapMemoryManager(220): Stopping 2023-05-04 10:55:40,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7. 2023-05-04 10:55:40,572 INFO [RS:2;jenkins-hbase4:42285] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-05-04 10:55:40,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7. after waiting 0 ms 2023-05-04 10:55:40,572 INFO [RS:2;jenkins-hbase4:42285] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-05-04 10:55:40,572 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-05-04 10:55:40,573 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-05-04 10:55:40,573 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-05-04 10:55:40,573 INFO [RS:2;jenkins-hbase4:42285] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:40,573 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7. 2023-05-04 10:55:40,573 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 468c84c2202e377e79810bff249e3ae7 1/1 column families, dataSize=78 B heapSize=488 B 2023-05-04 10:55:40,573 DEBUG [RS:0;jenkins-hbase4:37343] regionserver.HRegionServer(1505): Waiting on 468c84c2202e377e79810bff249e3ae7 2023-05-04 10:55:40,573 DEBUG [RS:2;jenkins-hbase4:42285] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4da1b00c to 127.0.0.1:53687 2023-05-04 10:55:40,573 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-05-04 10:55:40,573 DEBUG [RS:2;jenkins-hbase4:42285] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-04 10:55:40,574 INFO [RS:2;jenkins-hbase4:42285] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-05-04 10:55:40,574 INFO [RS:2;jenkins-hbase4:42285] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-05-04 10:55:40,574 INFO [RS:2;jenkins-hbase4:42285] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-05-04 10:55:40,574 INFO [RS:2;jenkins-hbase4:42285] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-05-04 10:55:40,574 INFO [RS:2;jenkins-hbase4:42285] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-05-04 10:55:40,574 DEBUG [RS:2;jenkins-hbase4:42285] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-05-04 10:55:40,574 DEBUG [RS:2;jenkins-hbase4:42285] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-05-04 10:55:40,574 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-05-04 10:55:40,574 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-05-04 10:55:40,574 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-05-04 10:55:40,574 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-05-04 10:55:40,574 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-05-04 10:55:40,574 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-05-04 10:55:40,759 INFO [AsyncFSWAL-0-hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d-prefix:jenkins-hbase4.apache.org,42285,1683197625528.meta] wal.AbstractFSWAL(1141): Slow sync cost: 183 ms, current pipeline: [DatanodeInfoWithStorage[127.0.0.1:44687,DS-ff8dcb33-2008-4522-bdfb-c059988b1bb3,DISK], DatanodeInfoWithStorage[127.0.0.1:38321,DS-0b13d23a-a532-49b1-93c9-373b659adc8e,DISK], DatanodeInfoWithStorage[127.0.0.1:34481,DS-e2fcb6ee-ea17-48bd-ab8e-961f13d8341b,DISK]] 2023-05-04 10:55:40,767 DEBUG [RS:1;jenkins-hbase4:33581] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/oldWALs 2023-05-04 10:55:40,767 INFO [RS:1;jenkins-hbase4:33581] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33581%2C1683197625489:(num 1683197627045) 2023-05-04 10:55:40,767 DEBUG [RS:1;jenkins-hbase4:33581] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-04 10:55:40,767 INFO [RS:1;jenkins-hbase4:33581] regionserver.LeaseManager(133): Closed leases 2023-05-04 10:55:40,768 INFO [RS:1;jenkins-hbase4:33581] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-05-04 10:55:40,768 INFO [RS:1;jenkins-hbase4:33581] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-05-04 10:55:40,768 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-05-04 10:55:40,768 INFO [RS:1;jenkins-hbase4:33581] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-05-04 10:55:40,768 INFO [RS:1;jenkins-hbase4:33581] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-05-04 10:55:40,773 INFO [RS:1;jenkins-hbase4:33581] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33581 2023-05-04 10:55:40,773 DEBUG [RS:0;jenkins-hbase4:37343] regionserver.HRegionServer(1505): Waiting on 468c84c2202e377e79810bff249e3ae7 2023-05-04 10:55:40,774 DEBUG [RS:2;jenkins-hbase4:42285] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-05-04 10:55:40,780 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/namespace/468c84c2202e377e79810bff249e3ae7/.tmp/info/3eeec2217f854592aa3304d83f9e82bf 2023-05-04 10:55:40,780 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:40,780 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:40,780 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-04 10:55:40,780 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-04 10:55:40,780 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-04 10:55:40,780 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33581,1683197625489 2023-05-04 10:55:40,781 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-04 10:55:40,782 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33581,1683197625489] 2023-05-04 10:55:40,782 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33581,1683197625489; numProcessing=1 2023-05-04 10:55:40,783 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,33581,1683197625489 already deleted, retry=false 2023-05-04 10:55:40,784 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,33581,1683197625489 expired; onlineServers=2 2023-05-04 10:55:40,785 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/namespace/468c84c2202e377e79810bff249e3ae7/.tmp/info/3eeec2217f854592aa3304d83f9e82bf as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/namespace/468c84c2202e377e79810bff249e3ae7/info/3eeec2217f854592aa3304d83f9e82bf 2023-05-04 10:55:40,788 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/namespace/468c84c2202e377e79810bff249e3ae7/info/3eeec2217f854592aa3304d83f9e82bf, entries=2, sequenceid=6, filesize=4.8 K 2023-05-04 10:55:40,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 468c84c2202e377e79810bff249e3ae7 in 216ms, sequenceid=6, compaction requested=false 2023-05-04 10:55:40,792 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/.tmp/info/be2378ee3d0d4fabb6b400b69613827a 2023-05-04 10:55:40,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/namespace/468c84c2202e377e79810bff249e3ae7/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-05-04 10:55:40,795 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for be2378ee3d0d4fabb6b400b69613827a 2023-05-04 10:55:40,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:55:40,796 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7. 2023-05-04 10:55:40,796 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 468c84c2202e377e79810bff249e3ae7: 2023-05-04 10:55:40,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1683197627461.468c84c2202e377e79810bff249e3ae7. 2023-05-04 10:55:40,810 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/.tmp/rep_barrier/3330994e6e52412c83525b37c830c718 2023-05-04 10:55:40,814 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3330994e6e52412c83525b37c830c718 2023-05-04 10:55:40,826 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/.tmp/table/d1059af3edb24eb5a354fefed3628b74 2023-05-04 10:55:40,829 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d1059af3edb24eb5a354fefed3628b74 2023-05-04 10:55:40,829 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/.tmp/info/be2378ee3d0d4fabb6b400b69613827a as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/info/be2378ee3d0d4fabb6b400b69613827a 2023-05-04 10:55:40,833 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for be2378ee3d0d4fabb6b400b69613827a 2023-05-04 10:55:40,833 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/info/be2378ee3d0d4fabb6b400b69613827a, entries=59, sequenceid=210, filesize=11.1 K 2023-05-04 10:55:40,834 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/.tmp/rep_barrier/3330994e6e52412c83525b37c830c718 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/rep_barrier/3330994e6e52412c83525b37c830c718 2023-05-04 10:55:40,837 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3330994e6e52412c83525b37c830c718 2023-05-04 10:55:40,837 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/rep_barrier/3330994e6e52412c83525b37c830c718, entries=24, sequenceid=210, filesize=7.5 K 2023-05-04 10:55:40,838 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/.tmp/table/d1059af3edb24eb5a354fefed3628b74 as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/table/d1059af3edb24eb5a354fefed3628b74 2023-05-04 10:55:40,841 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d1059af3edb24eb5a354fefed3628b74 2023-05-04 10:55:40,842 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/table/d1059af3edb24eb5a354fefed3628b74, entries=36, sequenceid=210, filesize=8.0 K 2023-05-04 10:55:40,842 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 268ms, sequenceid=210, compaction requested=false 2023-05-04 10:55:40,852 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-05-04 10:55:40,853 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-04 10:55:40,853 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-05-04 10:55:40,854 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-05-04 10:55:40,854 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-05-04 10:55:40,854 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-05-04 10:55:40,869 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-05-04 10:55:40,869 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-05-04 10:55:40,882 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-04 10:55:40,882 INFO [RS:1;jenkins-hbase4:33581] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,33581,1683197625489; zookeeper connection closed. 2023-05-04 10:55:40,882 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:33581-0x101e94d661e0002, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-04 10:55:40,883 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@121e8e88] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@121e8e88 2023-05-04 10:55:40,894 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-05-04 10:55:40,894 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-05-04 10:55:40,974 INFO [RS:0;jenkins-hbase4:37343] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,37343,1683197625438; all regions closed. 2023-05-04 10:55:40,974 INFO [RS:2;jenkins-hbase4:42285] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,42285,1683197625528; all regions closed. 2023-05-04 10:55:40,981 DEBUG [RS:0;jenkins-hbase4:37343] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/oldWALs 2023-05-04 10:55:40,981 DEBUG [RS:2;jenkins-hbase4:42285] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/oldWALs 2023-05-04 10:55:40,981 INFO [RS:2;jenkins-hbase4:42285] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42285%2C1683197625528.meta:.meta(num 1683197627191) 2023-05-04 10:55:40,981 INFO [RS:0;jenkins-hbase4:37343] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37343%2C1683197625438:(num 1683197627044) 2023-05-04 10:55:40,981 DEBUG [RS:0;jenkins-hbase4:37343] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-04 10:55:40,982 INFO [RS:0;jenkins-hbase4:37343] regionserver.LeaseManager(133): Closed leases 2023-05-04 10:55:40,982 INFO [RS:0;jenkins-hbase4:37343] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-05-04 10:55:40,982 INFO [RS:0;jenkins-hbase4:37343] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-05-04 10:55:40,982 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-05-04 10:55:40,982 INFO [RS:0;jenkins-hbase4:37343] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-05-04 10:55:40,983 INFO [RS:0;jenkins-hbase4:37343] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-05-04 10:55:40,983 INFO [RS:0;jenkins-hbase4:37343] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37343 2023-05-04 10:55:40,986 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:40,986 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37343,1683197625438 2023-05-04 10:55:40,986 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-04 10:55:40,988 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,37343,1683197625438] 2023-05-04 10:55:40,988 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,37343,1683197625438; numProcessing=2 2023-05-04 10:55:40,990 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,37343,1683197625438 already deleted, retry=false 2023-05-04 10:55:40,990 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,37343,1683197625438 expired; onlineServers=1 2023-05-04 10:55:40,992 DEBUG [RS:2;jenkins-hbase4:42285] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/oldWALs 2023-05-04 10:55:40,992 INFO [RS:2;jenkins-hbase4:42285] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42285%2C1683197625528:(num 1683197627045) 2023-05-04 10:55:40,992 DEBUG [RS:2;jenkins-hbase4:42285] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-04 10:55:40,992 INFO [RS:2;jenkins-hbase4:42285] regionserver.LeaseManager(133): Closed leases 2023-05-04 10:55:40,992 INFO [RS:2;jenkins-hbase4:42285] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-05-04 10:55:40,992 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-05-04 10:55:40,993 INFO [RS:2;jenkins-hbase4:42285] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42285 2023-05-04 10:55:40,994 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42285,1683197625528 2023-05-04 10:55:40,994 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-04 10:55:40,994 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42285,1683197625528] 2023-05-04 10:55:40,994 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42285,1683197625528; numProcessing=3 2023-05-04 10:55:40,997 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,42285,1683197625528 already deleted, retry=false 2023-05-04 10:55:40,997 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42285,1683197625528 expired; onlineServers=0 2023-05-04 10:55:40,997 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,40233,1683197624291' ***** 2023-05-04 10:55:40,997 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-05-04 10:55:40,997 DEBUG [M:0;jenkins-hbase4:40233] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@46b276d6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-05-04 10:55:40,998 INFO [M:0;jenkins-hbase4:40233] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,40233,1683197624291 2023-05-04 10:55:40,998 INFO [M:0;jenkins-hbase4:40233] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,40233,1683197624291; all regions closed. 2023-05-04 10:55:40,998 DEBUG [M:0;jenkins-hbase4:40233] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-04 10:55:40,998 DEBUG [M:0;jenkins-hbase4:40233] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-05-04 10:55:40,998 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-05-04 10:55:40,998 DEBUG [M:0;jenkins-hbase4:40233] cleaner.HFileCleaner(317): Stopping file delete threads 2023-05-04 10:55:40,998 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1683197626658] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1683197626658,5,FailOnTimeoutGroup] 2023-05-04 10:55:40,998 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1683197626657] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1683197626657,5,FailOnTimeoutGroup] 2023-05-04 10:55:40,998 INFO [M:0;jenkins-hbase4:40233] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-05-04 10:55:40,998 INFO [M:0;jenkins-hbase4:40233] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-05-04 10:55:40,998 INFO [M:0;jenkins-hbase4:40233] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-05-04 10:55:40,998 DEBUG [M:0;jenkins-hbase4:40233] master.HMaster(1512): Stopping service threads 2023-05-04 10:55:40,998 INFO [M:0;jenkins-hbase4:40233] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-05-04 10:55:40,999 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-05-04 10:55:40,999 ERROR [M:0;jenkins-hbase4:40233] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] 2023-05-04 10:55:40,999 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-05-04 10:55:40,999 INFO [M:0;jenkins-hbase4:40233] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-04 10:55:40,999 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-05-04 10:55:40,999 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-04 10:55:40,999 DEBUG [M:0;jenkins-hbase4:40233] zookeeper.ZKUtil(398): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-05-04 10:55:40,999 WARN [M:0;jenkins-hbase4:40233] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-05-04 10:55:40,999 INFO [M:0;jenkins-hbase4:40233] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-04 10:55:41,000 INFO [M:0;jenkins-hbase4:40233] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-05-04 10:55:41,000 DEBUG [M:0;jenkins-hbase4:40233] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-05-04 10:55:41,000 INFO [M:0;jenkins-hbase4:40233] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-04 10:55:41,000 DEBUG [M:0;jenkins-hbase4:40233] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-04 10:55:41,000 DEBUG [M:0;jenkins-hbase4:40233] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-05-04 10:55:41,000 DEBUG [M:0;jenkins-hbase4:40233] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-04 10:55:41,000 INFO [M:0;jenkins-hbase4:40233] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=607.95 KB heapSize=718.97 KB 2023-05-04 10:55:41,018 INFO [M:0;jenkins-hbase4:40233] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=607.95 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/8b5278577833445890979a8507b58e6f 2023-05-04 10:55:41,022 INFO [M:0;jenkins-hbase4:40233] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8b5278577833445890979a8507b58e6f 2023-05-04 10:55:41,023 DEBUG [M:0;jenkins-hbase4:40233] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/8b5278577833445890979a8507b58e6f as hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/8b5278577833445890979a8507b58e6f 2023-05-04 10:55:41,026 INFO [M:0;jenkins-hbase4:40233] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8b5278577833445890979a8507b58e6f 2023-05-04 10:55:41,026 INFO [M:0;jenkins-hbase4:40233] regionserver.HStore(1080): Added hdfs://localhost:44245/user/jenkins/test-data/1b77a26c-cdbc-ab5d-f437-939bf83e196d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/8b5278577833445890979a8507b58e6f, entries=156, sequenceid=1282, filesize=34.9 K 2023-05-04 10:55:41,027 INFO [M:0;jenkins-hbase4:40233] regionserver.HRegion(2948): Finished flush of dataSize ~607.95 KB/622539, heapSize ~718.95 KB/736208, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 27ms, sequenceid=1282, compaction requested=false 2023-05-04 10:55:41,027 INFO [M:0;jenkins-hbase4:40233] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-04 10:55:41,027 DEBUG [M:0;jenkins-hbase4:40233] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-05-04 10:55:41,032 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-05-04 10:55:41,032 INFO [M:0;jenkins-hbase4:40233] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-05-04 10:55:41,032 INFO [M:0;jenkins-hbase4:40233] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40233 2023-05-04 10:55:41,034 DEBUG [M:0;jenkins-hbase4:40233] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,40233,1683197624291 already deleted, retry=false 2023-05-04 10:55:41,090 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-04 10:55:41,090 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:37343-0x101e94d661e0001, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-04 10:55:41,090 INFO [RS:0;jenkins-hbase4:37343] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,37343,1683197625438; zookeeper connection closed. 2023-05-04 10:55:41,090 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1ccd3dd1] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1ccd3dd1 2023-05-04 10:55:41,190 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-04 10:55:41,190 INFO [M:0;jenkins-hbase4:40233] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,40233,1683197624291; zookeeper connection closed. 2023-05-04 10:55:41,190 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): master:40233-0x101e94d661e0000, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-04 10:55:41,291 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-04 10:55:41,291 DEBUG [Listener at localhost/35483-EventThread] zookeeper.ZKWatcher(600): regionserver:42285-0x101e94d661e0003, quorum=127.0.0.1:53687, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-04 10:55:41,291 INFO [RS:2;jenkins-hbase4:42285] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,42285,1683197625528; zookeeper connection closed. 2023-05-04 10:55:41,291 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@41099236] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@41099236 2023-05-04 10:55:41,291 INFO [Listener at localhost/35483] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-05-04 10:55:41,292 WARN [Listener at localhost/35483] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-05-04 10:55:41,295 INFO [Listener at localhost/35483] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-05-04 10:55:41,400 WARN [BP-2139253951-172.31.14.131-1683197620399 heartbeating to localhost/127.0.0.1:44245] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2139253951-172.31.14.131-1683197620399 (Datanode Uuid eabd0028-d500-4e17-b99c-cf8614a5876b) service to localhost/127.0.0.1:44245 2023-05-04 10:55:41,402 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/cluster_3e5b0dbc-215f-67e3-28fd-7825f44cb1a4/dfs/data/data5/current/BP-2139253951-172.31.14.131-1683197620399] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-04 10:55:41,402 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/cluster_3e5b0dbc-215f-67e3-28fd-7825f44cb1a4/dfs/data/data6/current/BP-2139253951-172.31.14.131-1683197620399] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-04 10:55:41,405 WARN [Listener at localhost/35483] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-05-04 10:55:41,408 INFO [Listener at localhost/35483] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-05-04 10:55:41,516 WARN [BP-2139253951-172.31.14.131-1683197620399 heartbeating to localhost/127.0.0.1:44245] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2139253951-172.31.14.131-1683197620399 (Datanode Uuid 4e8a05ef-1582-45ea-8d3b-86bbf8f33f91) service to localhost/127.0.0.1:44245 2023-05-04 10:55:41,517 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/cluster_3e5b0dbc-215f-67e3-28fd-7825f44cb1a4/dfs/data/data3/current/BP-2139253951-172.31.14.131-1683197620399] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-04 10:55:41,517 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/cluster_3e5b0dbc-215f-67e3-28fd-7825f44cb1a4/dfs/data/data4/current/BP-2139253951-172.31.14.131-1683197620399] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-04 10:55:41,519 WARN [Listener at localhost/35483] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-05-04 10:55:41,521 INFO [Listener at localhost/35483] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-05-04 10:55:41,625 WARN [BP-2139253951-172.31.14.131-1683197620399 heartbeating to localhost/127.0.0.1:44245] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-05-04 10:55:41,625 WARN [BP-2139253951-172.31.14.131-1683197620399 heartbeating to localhost/127.0.0.1:44245] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2139253951-172.31.14.131-1683197620399 (Datanode Uuid 9432e34f-f582-4227-a602-1b8f52f4fa10) service to localhost/127.0.0.1:44245 2023-05-04 10:55:41,626 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/cluster_3e5b0dbc-215f-67e3-28fd-7825f44cb1a4/dfs/data/data1/current/BP-2139253951-172.31.14.131-1683197620399] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-04 10:55:41,627 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/561fb62b-ad51-1307-3b42-cd6dedef1462/cluster_3e5b0dbc-215f-67e3-28fd-7825f44cb1a4/dfs/data/data2/current/BP-2139253951-172.31.14.131-1683197620399] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-04 10:55:41,650 INFO [Listener at localhost/35483] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-05-04 10:55:41,838 INFO [Listener at localhost/35483] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-05-04 10:55:42,076 INFO [Listener at localhost/35483] hbase.HBaseTestingUtility(1293): Minicluster is down